Hadoop Live node为0，无datanode进程

原创

迷乐

修改于 2020-06-01 11:32:14

1.5K0

修改于 2020-06-01 11:32:14

文章被收录于专栏：数据-迷之欢乐数据-迷之欢乐

问题：

用VirtualBox安装Hadoop伪集群，start-all.sh过程没有报错，如下：

hadoop1@master:~$ start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [master]
The authenticity of host 'master (192.168.56.100)' can't be established.
ECDSA key fingerprint is SHA256:5v0Pv8H46CIUWEJBviEE3+hdPhc7y4jMdy6Sotf6nSQ.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
master: Warning: Permanently added 'master,192.168.56.100' (ECDSA) to the list of known hosts.
master: starting namenode, logging to /usr/local/hadoop-2.10.0/logs/hadoop-hadoop1-namenode-master.out
datanode1: starting datanode, logging to /usr/local/hadoop-2.10.0/logs/hadoop-hadoop1-datanode-datanode1.out
datanode2: starting datanode, logging to /usr/local/hadoop-2.10.0/logs/hadoop-hadoop1-datanode-datanode2.out
datanode3: starting datanode, logging to /usr/local/hadoop-2.10.0/logs/hadoop-hadoop1-datanode-datanode3.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.10.0/logs/hadoop-hadoop1-secondarynamenode-master.out
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop-2.10.0/logs/yarn-hadoop1-resourcemanager-master.out
datanode1: starting nodemanager, logging to /usr/local/hadoop-2.10.0/logs/yarn-hadoop1-nodemanager-datanode1.out
datanode2: starting nodemanager, logging to /usr/local/hadoop-2.10.0/logs/yarn-hadoop1-nodemanager-datanode2.out
datanode3: starting nodemanager, logging to /usr/local/hadoop-2.10.0/logs/yarn-hadoop1-nodemanager-datanode3.out
hadoop1@master:~$ jps
3252 SecondaryNameNode
3014 NameNode
3374 ResourceManage
3631 Jps

http://master:8088显示有3个datanode，但http://master:50070显示Live nodes为0.

查看子节点进程，发现没有datanode进程：

hadoop1@master:~$ ssh datanode1
hadoop1@datanode1:~$ jps
1915 Jps
1740 NodeManage
hadoop1@datanode1:~$ exit

解决：

查看子节点/usr/local/hadoop-2.10.0/logs/hadoop-hadoop1-datanode-datanode3.log日志文件，搜索WARN信息，看到：

2020-05-22 12:12:17,690 WARN org.apache.hadoop.hdfs.server.datanode.checker.StorageLocationChecker: Exception checking StorageLocation [DISK]file:/usr/loacal/hadoop-2.10.0/hadoop_data/hdfs/datanode java.io.FileNotFoundException: File file:/usr/loacal/hadoop-2.10.0/hadoop_data/hdfs/datanode does not exist

原来是目录错误，hdfs-site.xml设置的与实际目录/usr/local/hadoop-2.10.0/hadoop_data/hdfs/datanode（local误打成loacal）

<property>
        <name>dfs.datanode.data.dir</name>
        <value> file:/usr/loacal/hadoop-2.10.0/hadoop_data/hdfs/datanode</value>
</property>

修正各节点的hdfs-site.xml文件

<property>
        <name>dfs.datanode.data.dir</name>
        <value> file:/usr/local/hadoop-2.10.0/hadoop_data/hdfs/datanode</value>
</property>

并保存：

hadoop1@datanode3:~$ sudo nano /usr/local/hadoop-2.10.0/etc/hadoop/hdfs-site.xml 
hadoop1@datanode3:~$ ssh datanode1
hadoop1@datanode1:~$ sudo nano /usr/local/hadoop-2.10.0/etc/hadoop/hdfs-site.xml 
hadoop1@datanode1:~$ exit
hadoop1@datanode3:~$ ssh datanode2
hadoop1@datanode2:~$ sudo nano /usr/local/hadoop-2.10.0/etc/hadoop/hdfs-site.xml
hadoop1@datanode2:~$ exit
hadoop1@master:~$ sudo nano /usr/local/hadoop-2.10.0/etc/hadoop/hdfs-site.xml
hadoop1@master:~$ exit

在master终端中执行stop-all.sh，再执行start-all.sh，打开http://master:50070后发现已可以正常显示datanode情况.