hadoop - 由于输入字符串的 java.lang.NumberFormatException 导致 Apache Tez 作业失败:“30s”
问题描述
我正在尝试在 tez 上对 Apache hive 执行查询,但不知何故我收到如下错误,我不知道如何解决它
- 阿帕奇 Hadoop 3.1.1
- 阿帕奇蜂巢 3.1.0
- Apache Tez 0.9.1
我的 tez-site.xml
<configuration>
<property>
<name>tez.lib.uris</name>
<value>hdfs://localhost:8020/apps/apache-tez-0.9.1-bin/share/tez.tar.gz</value>
</property>
<property>
<name>tez.staging-dir</name>
<value>/tmp/${user.name}/staging</value>
</property>
<configuration>
020-04-22 21:08:55,530 [INFO] [main] |shim.HadoopShimsLoader|: Trying to locate HadoopShimProvider for hadoopVersion=2.7.0, majorVersion=2, minorVersion=7
2020-04-22 21:08:55,531 [INFO] [main] |shim.HadoopShimsLoader|: Picked HadoopShim org.apache.tez.hadoop.shim.HadoopShim27, providerName=org.apache.tez.hadoop.shim.HadoopShim25_26_27Provider, overrideProviderViaConfig=null, hadoopVersion=2.7.0, majorVersion=2, minorVersion=7
2020-04-22 21:08:55,551 [INFO] [main] |app.DAGAppMaster|: AM Level configured TaskSchedulers: [0:TezYarn:null],[1:TezUber:null]
2020-04-22 21:08:55,551 [INFO] [main] |app.DAGAppMaster|: AM Level configured ContainerLaunchers: [0:TezYarn:null],[1:TezUber:null]
2020-04-22 21:08:55,551 [INFO] [main] |app.DAGAppMaster|: AM Level configured TaskCommunicators: [0:TezYarn:null],[1:TezUber:null]
2020-04-22 21:08:55,551 [INFO] [main] |app.DAGAppMaster|: Comparing client version with AM version, clientVersion=0.9.1, AMVersion=0.9.1
2020-04-22 21:08:55,633 [INFO] [main] |service.AbstractService|: Service org.apache.tez.dag.app.DAGAppMaster failed in state INITED; cause: java.lang.NumberFormatException: For input string: "30s"
java.lang.NumberFormatException: For input string: "30s"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1311)
at org.apache.hadoop.hdfs.DFSClient$Conf.<init>(DFSClient.java:502)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:637)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.tez.common.TezCommonUtils.getTezBaseStagingPath(TezCommonUtils.java:87)
at org.apache.tez.common.TezCommonUtils.getTezSystemStagingPath(TezCommonUtils.java:146)
at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:492)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
2020-04-22 21:08:55,636 [WARN] [main] |service.AbstractService|: When stopping the service org.apache.tez.dag.app.DAGAppMaster : java.lang.NullPointerException
java.lang.NullPointerException
解决方案
尝试在 hdfs-site.xml 中添加此属性
<property>
<name>dfs.client.datanode-restart.timeout</name>
<value>30</value>
</property>
推荐阅读
- sql - 如何在 sql server 中进行插值?
- javascript - 分配给变量时使用 Jquery DataTable
- javascript - 如何访问已经验证的数据?
- python-3.x - 在子进程中使用 Popen 实用程序运行系统命令
- enums - Corona SDK 对矮人要塞之类的游戏是否足够快?
- c# - 将自定义参数添加到 IdentityServer4 身份验证请求
- node.js - antlr4-tool 在 Win10 中失败:错误:命令失败:哪个 java
- ios - 在 NSTextStorage 范围内崩溃
- json - 如何避免 api 响应结果中的“可选”?
- c - LinkedList 打印相同的值