hadoop 单机伪分布式安装步骤

Michael阿明

发布于 2021-09-06 10:09:18

2600

发布于 2021-09-06 10:09:18

文章被收录于专栏：Michael阿明学习之路

文章目录

环境 Centos7

参考：

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html

https://blog.csdn.net/bingduanlbd/article/details/51874195

1. 安装 Java

注意版本要求：

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions

Java 安装：https://cloud.tencent.com/developer/article/1872757

2. 配置SSH无密码登录

ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys
ssh localhost

第一条输入后，一直回车，无密码

3. 下载 hadoop

下载地址：https://hadoop.apache.org/releases.html，选择 binary downloads

解压到路径，我的是 /opt/hadoop-3.3.0/

tar -zxvf 包名

4. 配置环境变量

vim ~/.bashrc

贴下我的内容：

export PATH="~/anaconda3/bin:$PATH"

export JAVA_HOME=/opt/jdk1.8.0_281/
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

export HADOOP_HOME=/opt/hadoop-3.3.0/
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

export PATH=$PATH:/home/dnn/pycharm-2020.2.3/bin
export PATH=${PATH}:${HADOOP_HOME}/bin
export PATH=${PATH}:${JAVA_HOME}/bin
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root
#export HADOOP_ROOT_LOGGER=DEBUG,console
export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native