时间戳

首页 > 解决方案 > 安装 hadoop 后出现“WARN hdfs.DFSClient: DataStreamer Exception”

hadoop - 安装 hadoop 后出现“WARN hdfs.DFSClient: DataStreamer Exception”

问题描述

我已经在独立模式下安装了 Hadoop 2.7.7。

启动服务：

Starting namenodes on [hnode1]
hnode1: starting namenode, logging to /opt/hadoop/logs/hadoop-hduser-namenode-hnode1.out
localhost: starting datanode, logging to /opt/hadoop/logs/hadoop-hduser-datanode-hnode1.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /opt/hadoop/logs/hadoop-hduser- secondarynamenode-hnode1.out

starting yarn daemons
starting resourcemanager, logging to /opt/hadoop/logs/yarn-hduser-resourcemanager-hnode1.out
localhost: starting nodemanager, logging to /opt/hadoop/logs/yarn-hduser-nodemanager-hnode1.out

但是，在从示例文档运行一个简单的 mapreduce 作业时，会给出以下内容：

$HADOOP_INSTALL/bin/hadoop jar /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.7.jar pi 10 10

WARN hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/hduser/QuasiMonteCarlo_1533379418377_2070416414/in/part0 could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.
    at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1620)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3135)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3059)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:725)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:493)
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2211)
    .
    .
    .

我对输出的这一部分特别困扰： There are 0 datanode(s) running and no node(s) are excluded in this operation.在输出中。但是，jps输出：

17795 Jps
15604 DataNode
17350 NameNode
15994 NodeManager
15898 ResourceManager
17548 SecondaryNameNode

我怎样才能解决这个问题 ？

标签： hadoop

解决方案

推荐阅读