时间戳

首页 > 解决方案 > 错误：java.lang.RuntimeException：java.lang.ClassNotFoundException：找不到类 WordCountExample$Map

java - 错误：java.lang.RuntimeException：java.lang.ClassNotFoundException：找不到类 WordCountExample$Map

问题描述

我正在使用 ubuntu 系统并尝试运行 wordcount.jar 程序。不幸的是，我遇到以下错误 -

错误：java.lang.RuntimeException：java.lang.ClassNotFoundException：找不到类 WordCountExample$Map

我已经按原样更新了类路径 -

job.setJarByClass(WordCountExample.class); 和 jobconf.setJarByClass(WordCountExample.class);

这些都不起作用。不知道有什么问题。请在这方面分享您的外籍人士。

import java.io.IOException;
import java.util.*;

import org.apache.hadoop.fs.Path;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;

/** * @作者http://www.devinline.com */

public class WordCountExample {
/* Map class which job will use and execute it map method */
public static class Map extends
Mapper<LongWritable, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();

public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()) {
word.set(tokenizer.nextToken());
context.write(word, one);
}}}

/* Reduce class which job will use and execute it reduce method */
public static class Reduce extends
Reducer<Text, IntWritable, Text, IntWritable> {

public void reduce(Text key, Iterable<IntWritable> values,
Context context) throws IOException, InterruptedException {
int sum = 0;
for (IntWritable val : values) {
sum += val.get();
}
context.write(key, new IntWritable(sum));
}}

public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();

/* Created a job with name wordCountExample */
Job job = Job.getInstance(conf, "wordCountExample");
job.setJarByClass(WordCountExample.class);  

/*
* Handler string and int in hadoop way: for string hadoop uses Text
* class and for int uses IntWritable */
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);

/*
 * Configure map and reducer class, based on which it uses map and
    /* reduce method
     */
job.setMapperClass(Map.class);
job.setReducerClass(Reduce.class);

    /* Input and output format set as TextInputFormat */
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);

    /* addInputPath - passes input file path to job */
FileInputFormat.addInputPath(job, new Path(args[0]));
    /* setOutputPath - passes output path to job */
FileOutputFormat.setOutputPath(job, new Path(args[1]));

    /* Submit the job to the cluster and wait for it to finish. */
System.exit(job.waitForCompletion(true) ? 1 : 0);
}}

hadoop jar /home/nahmed/WordcountSample.jar WordCountExample /user/nahmed/pg20417.txt /user/nahmed/WCoutput 19/08/15 21:31:12 INFO client.RMProxy：在 it066431.massey.ac 连接到 ResourceManager。 nz/130.123.248.83:8050

19/08/15 21:31:12 INFO client.AHSProxy：连接到应用程序历史服务器 it066431.massey.ac.nz/130.123.248.83:10200 19/08/15 21:31:12 WARN mapreduce.JobResourceUploader：Hadoop未执行命令行选项解析。实现 Tool 接口并使用 ToolRunner 执行您的应用程序来解决这个问题。19/08/15 21:31:12 WARN mapreduce.JobResourceUploader：没有设置作业 jar 文件。可能找不到用户类。请参阅 Job 或 Job#setJar(String)。19/08/15 21:31:12 INFO input.FileInputFormat：要处理的总输入路径：1 19/08/15 21:31:12 INFO mapreduce.JobSubmitter：拆分数：1 19/08/15 21:31 ：12 INFO mapreduce.JobSubmitter：提交作业令牌：job_1562128011754_0026 19/08/15 21:31:13 INFO mapred.YARNRunner：作业 jar 不存在。不将任何 jar 添加到资源列表中。19/08/15 21:31:13 信息 impl.YarnClientImpl:http://it066431.massey.ac.nz:8088/proxy/application_1562128011754_0026/ 19/08/15 21:31:13 INFO mapreduce.Job：运行作业：job_1562128011754_0026 19/08/15 21:31:17 INFO mapreduce.Job：作业 job_1562128011754_0026 在 uber 模式下运行：假 19/08/15 21:31 ：17 INFO mapreduce.Job：map 0% reduce 0% 19/08/15 21:31:20 INFO mapreduce.Job：任务 ID：attempt_1562128011754_0026_m_000000_0，状态：FAILED 错误：java.lang.RuntimeException：java.lang.ClassNotFoundException：在 org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186) 的 org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186) 中找不到类 WordCountExample$Map .hadoop.mapred.MapTask.runNewMapper(MapTask.java:745) 在 org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) 在 org.apache.hadoop.mapred.YarnChild$2.run(YarnChild. java:170) 在 java.security.AccessController。doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.mapred。 YarnChild.main(YarnChild.java:164) 原因：java.lang.ClassNotFoundException: Class WordCountExample$Map not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2134) at org.apache.hadoop .conf.Configuration.getClass(Configuration.java:2226) ... 还有 8 个getClassByName(Configuration.java:2134) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2226) ... 还有 8 个getClassByName(Configuration.java:2134) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2226) ... 还有 8 个

容器被 ApplicationMaster 杀死。容器应要求被杀死。退出代码为 143 容器以非零退出代码 143 退出

2015 年 8 月 19 日 21:31:23 信息 mapreduce.Job：任务 ID：尝试_1562128011754_0026_m_000000_1，状态：失败错误：java.lang.RuntimeException：java.lang.ClassNotFoundException：在 org.apache.hadoop 中找不到类 WordCountExample$Map。 conf.Configuration.getClass(Configuration.java:2228) 在 org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186) 在 org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java: 745) 在 org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) 在 org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170) 在 java.security.AccessController.doPrivileged( Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.mapred.YarnChild。主要（YarnChild.java:164) 原因：java.lang.ClassNotFoundException: Class WordCountExample$Map not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2134) at org.apache.hadoop.conf.Configuration.getClass (Configuration.java:2226) ... 8 更多

容器被 ApplicationMaster 杀死。容器应要求被杀死。退出代码为 143 容器以非零退出代码 143 退出

2015 年 8 月 19 日 21:31:26 信息 mapreduce.Job：任务 ID：尝试_1562128011754_0026_m_000000_2，状态：失败错误：java.lang.RuntimeException：java.lang.ClassNotFoundException：在 org.apache.hadoop 中找不到类 WordCountExample$Map。 conf.Configuration.getClass(Configuration.java:2228) 在 org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186) 在 org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java: 745) 在 org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) 在 org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170) 在 java.security.AccessController.doPrivileged( Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.mapred.YarnChild。主要（YarnChild.java:164) 原因：java.lang.ClassNotFoundException: Class WordCountExample$Map not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2134) at org.apache.hadoop.conf.Configuration.getClass (Configuration.java:2226) ... 8 更多

容器被 ApplicationMaster 杀死。容器应要求被杀死。退出代码为 143 容器以非零退出代码 143 退出

19/08/15 21:31:31 INFO mapreduce.Job: map 100% reduce 100% 19/08/15 21:31:31 INFO mapreduce.Job: Job job_1562128011754_0026 failed with state FAILED due to: Task failed task_1562128011754_0026_m_00000由于任务失败。failedMaps:1 failedReduces:0

19/08/15 21:31:31 INFO mapreduce.Job: Counters: 13 Job Counters Failed map tasks=4 Killed reduce tasks=1 Launched map tasks=4 Other local map tasks=3 Data-local map tasks=1 Total time所有map在占用槽中花费的时间（ms）=53207 所有reduce在占用槽中花费的总时间（ms）=0 所有map任务花费的总时间（ms）=7601 所有reduce任务花费的总时间（ms）=0所有map任务花费的总vcore毫秒=7601所有reduce任务花费的总vcore毫秒=0所有map任务花费的总兆字节毫秒=54483968所有reduce任务花费的总兆字节毫秒=0

标签： javahadoop

解决方案

您有两个嵌套类：Map 和 Reduce。该错误表明java无法找到它们。jar /home/nahmed/WordcountSample.jar 很可能不包含它们。验证执行：

 jar -tf  /home/nahmed/WordcountSample.jar

如果它们没有包含在 jar 中，我认为问题在于您的编译方式。

推荐阅读