hadoop - Apache Pig 问题 - 使用顺序限制数据不起作用
问题描述
我有个问题。当我加载输入数据时,然后我计算数据,按名称排序数据,然后我尝试获取前 3 个数据但有一些问题。这是日志文件转储数据;
(Tainan,57)
(Taipei,121)
(Hsinchu,53)
(Hualien,44)
(Taitung,50)
(Taoyuan,72)
(Pingtung,59)
(Taichung,63)
(Kaohsiung,89)
然后使用 $1 desc 的订单数据:
(Taipei,121)
(Kaohsiung,89)
(Taoyuan,72)
(Taichung,63)
(Pingtung,59)
(Tainan,57)
(Hsinchu,53)
(Taitung,50)
(Hualien,44)
但是当我尝试使用命令时: data = limit data 3; 并倾倒它。它有问题。
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
3.2.2 0.17.0 hadoop 2021-05-05 19:49:54 2021-05-05 19:49:55 GROUP_BY,ORDER_BY,LIMIT
Some jobs have failed! Stop running all dependent jobs
Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTime AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTimMedianReducetime Alias Feature Outputs
job_local586139516_0006 1 1 n/a n/a n/a n/a n/a n/a n/a n/a q11 SAMPLER
job_local708055803_0005 1 1 n/a n/a n/a n/a n/a n/a n/a n/a a,c0,l,q10 GROUP_BY
Failed Jobs:
JobId Alias Feature Message Outputs
job_local1801700609_0007 q11 ORDER_BY,COMBINER Message: Job failed!
Input(s):
Successfully read 608 records from: "file:///home/hadoop/mid/profile.txt"
Output(s):
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local708055803_0005 -> job_local586139516_0006,
job_local586139516_0006 -> job_local1801700609_0007,
job_local1801700609_0007 -> null,
null
2021-05-05 19:49:55,716 [main] WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl - JobTracker metrics system already initialized!
2021-05-05 19:49:55,718 [main] WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl - JobTracker metrics system already initialized!
2021-05-05 19:49:55,720 [main] WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl - JobTracker metrics system already initialized!
2021-05-05 19:49:55,736 [main] WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl - JobTracker metrics system already initialized!
2021-05-05 19:49:55,745 [main] WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl - JobTracker metrics system already initialized!
2021-05-05 19:49:55,749 [main] WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl - JobTracker metrics system already initialized!
2021-05-05 19:49:55,755 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Some jobs have failed! Stop running all dependent jobs
2021-05-05 19:49:55,758 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias q12
Details at logfile: /home/hadoop/mid/pig_1620215111006.log
解决方案
推荐阅读
- java - 将 Java 变成 Android 应用程序。在加载时添加了 elif 和项目崩溃
- react-native - React Native:我如何禁用按钮?
- python-3.x - 当“每月”或“每年”拼写错误时,无法从函数中获得回报
- python - Tensorflow,无法弄清楚我的输入和标签有什么形状
- f# - F#:“类型 X = Z 的 Y”的含义
- javascript - 如何在不上传到NPM的情况下模仿我的库是节点模块?
- algorithm - 从子字符串列表构造目标字符串
- java - 运行 gradle 需要很长时间并且仍在运行
- r - 使用 Rmarkdown 编织到 pdf 时遇到麻烦
- python - AttributeError: _Connection__connection 使用 sqlalchemy 提交会话时