java - Map reduce 作业执行但不产生输出

问题描述

请寻求帮助。mapreduce 作业执行但不产生输出。这是一个计算文件中总字数的简单程序。我开始非常简单地确保它适用于一个 txt 文件，该文件有一行包含以下内容：

小国第二大国第二小食品出口国第二第二第二

不幸的是，它没有，任何关于下一步看哪里的建议将不胜感激。我已经剪切并粘贴了输出日志的最后一点。

File System Counters
    FILE: Number of bytes read=890
    FILE: Number of bytes written=947710
    FILE: Number of read operations=0
    FILE: Number of large read operations=0
    FILE: Number of write operations=0
Map-Reduce Framework
    Map input records=1
    Map output records=1
    Map output bytes=87
    Map output materialized bytes=95
    Input split bytes=198
    Combine input records=0
    Combine output records=0
    Reduce input groups=1
    Reduce shuffle bytes=95
    Reduce input records=1
    Reduce output records=1
    Spilled Records=2
    Shuffled Maps =1
    Failed Shuffles=0
    Merged Map outputs=1
    GC time elapsed (ms)=7
    Total committed heap usage (bytes)=468713472
Shuffle Errors
    BAD_ID=0
    CONNECTION=0
    IO_ERROR=0
    WRONG_LENGTH=0
    WRONG_MAP=0
    WRONG_REDUCE=0
File Input Format Counters 
    Bytes Read=82
File Output Format Counters 
    Bytes Written=97
Process finished with exit code 0

    public class Map extends Mapper<LongWritable, Text, Text, 
        IntWritable>{

    @Override
    public void map(LongWritable key, Text value, Context context)
            throws IOException, InterruptedException {

        String line = value.toString();
        String[] datas = line.split("\t");

            for(String data: datas) {
                Text outputKey = new Text(data);
                IntWritable outputValue = new IntWritable();
                context.write(outputKey, outputValue);
            }
    }
    }

public class Reduce extends Reducer<Text, IntWritable, Text, 
    IntWritable> {

    @Override
    public void reduce(final Text outputKey,
                       final Iterable<IntWritable> values,
                       final Context context)
            throws IOException, InterruptedException {

        int sum = 0;
        for(IntWritable value : values)
        {
            sum += value.get();
        }
        context.write(outputKey, new IntWritable(sum));
    }
}

public class Main extends Configured implements Tool {


    @Override
    public int run(String[] args) throws Exception {

        Job job = Job.getInstance(getConf());
        job.setJobName("WordCount");
        job.setJarByClass(Main.class);

        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);

        job.setMapperClass(Map.class);
        job.setReducerClass(Reduce.class);

        Path inputFilePath = new Path("/Users/francesco/input/input.txt");
        Path outputFilePath = new Path("/Users/francesco/output/first");

        FileInputFormat.addInputPath(job, inputFilePath);
        FileOutputFormat.setOutputPath(job, outputFilePath);

        return job.waitForCompletion(true) ? 0 : 1;
    }

    public static void main(String[] args) throws Exception{
        int exitCode = ToolRunner.run(new Main(), args);
        System.exit(exitCode);
    }
}

标签： javahadoopmapreduce

java - Map reduce 作业执行但不产生输出

问题描述

解决方案

推荐阅读