hadoop - 具有 HDFS 高可用性的 Hadoop 集群

Question

将 HDFS 元数据从活动名称节点复制到备用名称节点。bin/hdfs namenode -bootstrapStandby 19/09/01 02:40:38 INFO namenode.NameNode：为 [TERM、HUP、INT] 注册的 UNIX 信号处理程序 19/09/01 02:40:38 INFO namenode.NameNode：createNameNode [ -bootstrapStandby] 19/09/01 02:40:39 WARN common.Util：路径 /home/kenny/hadoop-2.7.3/namenode 应在配置文件中指定为 URI。请更新 hdfs 配置。19/09/01 02:40:39 WARN common.Util：路径 /home/kenny/hadoop-2.7.3/namenode 应在配置文件中指定为 URI。请更新 hdfs 配置。19/09/01 02:40:41 INFO ipc.Client：重试连接到服务器：node-master-2/192.168.1.170:9000。已尝试 0 次；重试策略是 RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 19/09/01 02:40：42 INFO ipc.Client：重试连接到服务器：node-master-2/192.168.1.170:9000。已尝试 1 次；重试策略是 RetryUpToMaximumCoun

核心站点设置
core-site.xml

<property>
    <name>fs.defaultFS</name>
    <value>hdfs://ha-cluster</value>
    </property>
    <property>
    <name>dfs.journalnode.edits.dir</name>
    <value>/home/kenny/hadoop-2.7.3/jn</value>
    </property>

hdfs-site 设置
hdfs-site.xml

<property>
    <name>dfs.namenode.name.dir</name>
    <value>/home/kenny/hadoop-2.7.3/namenode</value>
    </property>
    <property>
    <name>dfs.replication</name>
    <value>2</value>
    </property>
    <property>
    <name>dfs.permission</name>
    <value>false</value>
    </property>
    <property>
    <name>dfs.nameservices</name>
    <value>ha-cluster</value>
    </property>
    <property>
    <name>dfs.ha.namenodes.ha-cluster</name>
    <value>node-master,node-master-2</value>
    </property>
    <property>
    <name>dfs.namenode.rpc-address.ha-cluster.node-master</name>
    <value>node-master:9000</value>
    </property>
    <property>
    <name>dfs.namenode.rpc-address.ha-cluster.node-master-2</name>
    <value>node-master-2:9000</value>
    </property>
    <property>
    <name>dfs.namenode.http-address.ha-cluster.node-master</name>
    <value>node-master:50070</value>
    </property>
    <property>
    <name>dfs.namenode.http-address.ha-cluster.node-master-2</name>
    <value>node-master-2:50070</value>
    </property>
    <property>
    <name>dfs.namenode.shared.edits.dir</name>
    <value>qjournal://node-master:8485;node-master-2:8485;node1:8485/ha-cluster</value>
    </property>
    <property>
    <name>dfs.client.failover.proxy.provider.ha-cluster</name>
    <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>
    <property>
    <name>dfs.ha.automatic-failover.enabled</name>
    <value>true</value>
    </property>
    <property>
    <name>ha.zookeeper.quorum</name>
    <value>node-master:2181,node-master-2:2181,node1:2181 </value>
    </property>
    <property>
    <name>dfs.ha.fencing.methods</name>
    <value>sshfence</value>
    </property>
    <property>
    <name>dfs.ha.fencing.ssh.private-key-files</name>
    <value>/home/kenny/.ssh/id_rsa</value>
    </property>

标签： hadoophdfscluster-computinghigh-availability