安装 Ambari 2.6.0 + HDP 2.6.3 搭建Hadoop集群（CentOS7）简明版

Yunjie Ge

发布于 2022-04-24 09:24:48

9460

发布于 2022-04-24 09:24:48

文章被收录于专栏：数据库与编程

三台虚拟机

72.16.243.146 db_server_01

72.16.243.147 db_server_02

72.16.243.148 db_server_03

设置主机名

第1台：

hostnamectl set-hostname db_server_01

第2台：

hostnamectl set-hostname db_server_02

第3台：

hostnamectl set-hostname db_server_03

设置hosts，所有节点都设置

vi /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

172.16.243.146 db_server_01 db_server_01.hadoop

172.16.243.147 db_server_02 db_server_02.hadoop

172.16.243.148 db_server_03 db_server_03.hadoop

每台节点里配置全限定域名(FQDN)

第1台：

vi /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=db_server_01.hadoop

第2台：

vi /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=db_server_02.hadoop

第3台：

vi /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=db_server_03.hadoop

配置SSH免密码登录

1、在主节点（第1台）执行

ssh-keygen

cd ~/.ssh/

cat id_rsa.pub >>authorized_keys

chmod 600 ~/.ssh

chmod 600 ~/.ssh/authorized_keys

2、在所有从节点（第2、3台）执行

mkdir ~/.ssh/

3、在主节点执行

scp /root/.ssh/authorized_keys root@172.16.243.147:/root/.ssh/authorized_keys

scp /root/.ssh/authorized_keys root@172.16.243.148:/root/.ssh/authorized_keys

开启NTP服务，所有节点都需要执行

yum install ntp

systemctl is-enabled ntpd

systemctl enable ntpd

关闭防火墙，所有机器都执行

systemctl disable firewalld

systemctl stop firewalld

关闭SELinux，所有机器都执行

#临时关闭

setenforce 0

#永久关闭

vi /etc/sysconfig/selinux

SELINUX=disabled

安装HTTP 服务器（第1台），允许 http 服务通过防火墙(永久)

yum install httpd

firewall-cmd --add-service=http

firewall-cmd --permanent --add-service=http

添加 Apache 服务到系统层使其随系统自动启动

systemctl start httpd.service

systemctl enable httpd.service

安装本地源制作相关工具

yum install yum-utils createrepo

创建目录

cd /var/www/html/

mkdir ambari

mkdir HDP-UTILS

cd /var/www/html/ambari/

下载Ambari 2.6.0

wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.0.0/ambari-2.6.0.0-centos7.tar.gz

下载HDP 2.6.3

wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.3.0/HDP-2.6.3.0-centos7-rpm.tar.gz

下载HDP-UTILS 1.1.0.21

wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz

解压文件

tar -zxvf ambari-2.6.0.0-centos7.tar.gz -C /var/www/html/ambari/

tar -zxvf HDP-2.6.3.0-centos7-rpm.tar.gz -C /var/www/html/ambari/

tar -zxvf HDP-UTILS-1.1.0.21-centos7.tar.gz -C /var/www/html/ambari/HDP-UTILS/

测验证httpd网站是否可用

http://172.16.243.146/ambari/

安装Mysql数据库

yum install mariadb-server

systemctl start mariadb

systemctl enable mariadb

数据库初始化设置

mysql_secure_installation

Enter current password for root (enter for none):<–初次运行直接回车

Set root password? [Y/n] <– 是否设置root用户密码，输入y并回车或直接回车

New password: <– 设置root用户的密码

Re-enter new password: <– 再输入一次你设置的密码

Remove anonymous users? [Y/n] <– 是否删除匿名用户，回车

Disallow root login remotely? [Y/n] <–是否禁止root远程登录,回车,

Remove test database and access to it? [Y/n] <– 是否删除test数据库，回车

Reload privilege tables now? [Y/n] <– 是否重新加载权限表，回车

初始化MariaDB完成，接下来测试登录，输入密码能正常登陆就完成了

mysql -uroot -p

登录数据库root用户执行下面语句：

> create database ambari character set utf8 ;

> CREATE USER 'ambari'@'%'IDENTIFIED BY 'Ambari-123';

> GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';

> FLUSH PRIVILEGES;

> create database hive character set utf8 ;

> CREATE USER 'hive'@'%'IDENTIFIED BY 'Hive-123';

> GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';

> FLUSH PRIVILEGES;

> create database oozie character set utf8 ;

> CREATE USER 'oozie'@'%'IDENTIFIED BY 'Oozie-123';

> GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';

> FLUSH PRIVILEGES;

安装mysql jdbc 驱动

yum install mysql-connector-java

配置ambari、HDP、HDP-UTILS的本地源

cd /etc/yum.repos.d/

vi ambari.repo

#VERSION_NUMBER=2.6.0.0-267

[ambari-2.6.0.0]

name=ambari Version - ambari-2.6.0.0

baseurl=http://172.16.243.146/ambari/ambari/centos7/2.6.0.0-267

gpgcheck=1

gpgkey=http://172.16.243.146/ambari/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

vi hdp.repo

#VERSION_NUMBER=2.6.3.0-235

[HDP-2.6.3.0]

name=HDP Version - HDP-2.6.3.0

baseurl=http://172.16.243.146/ambari/HDP/centos7/2.6.3.0-235

gpgcheck=1

gpgkey=http://172.16.243.146/ambari/HDP/centos7/2.6.3.0-235/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

[HDP-UTILS-1.1.0.21]

name=HDP Utils Version - HDP-UTILS-1.1.0.21

baseurl=http://172.16.243.146/ambari/HDP-UTILS

gpgcheck=1

gpgkey=http://172.16.243.146/ambari/HDP-UTILS/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

添空yum缓存

yum clean all

yum list update

yum makecache

yum repolist

安装JDK

http://www.oracle.com/technetwork/cn/java/javase/downloads/jdk8-downloads-2133151-zhs.html

tar -zxvf jdk-8u152-linux-x64.tar.gz -C /opt/java/

卸载OpenJDK

rpm -qa|grep java

rpm -e --nodeps java-1.7.0-openjdk-1.7.0.9-2.3.4.1.el6_3.i686

设置环境变量

vi /etc/profile

export JAVA_HOME=/opt/java/jdk1.8.0_152

export CLASSPATH=.:JAVA_HOME/lib/dt.jar:JAVA_HOME/lib/tools.jar

PATH=PATH:HOME/bin:

source /etc/profile

安装Ambari

yum install ambari-server

配置Ambari

ambari-server setup

#下面是配置执行流程，按照提示操作

Using python /usr/bin/python

Setup ambari-server

Checking SELinux...

SELinux status is 'enabled'

SELinux mode is 'permissive'

WARNING: SELinux is set to 'permissive' mode and temporarily disabled.

OK to continue [y/n] (y)? y

Customize user account for ambari-server daemon [y/n] (n)? y

Enter user account for ambari-server daemon (root):ambari

Adjusting ambari-server permissions and ownership...

Checking firewall status...

Checking JDK...

[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8

[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7

[3] Custom JDK

==========================================================================

Enter choice (1): 3

WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.

WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.

Path to JAVA_HOME: /opt/java/jdk1.8.0_152

Validating JDK on Ambari Server...done.

Completing setup...

Configuring database...

Enter advanced database configuration [y/n] (n)? y

===========================================================================

Choose one of the following options:

[1] - PostgreSQL (Embedded)

[2] - Oracle

[3] - MySQL

[4] - PostgreSQL

[5] - Microsoft SQL Server (Tech Preview)

[6] - SQL Anywhere

===========================================================================

Enter choice (3): 3

Hostname (localhost):

Port (3306):

Database name (ambari):

Username (ambari):

Enter Database Password (bigdata): Ambari-123

Re-enter password: Ambari-123

Configuring ambari database...

Configuring remote database connection properties...

WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

Proceed with configuring remote database connection properties [y/n] (y)? y

Extracting system views...

ambari-admin-2.6.0.0.267.jar

...........

Adjusting ambari-server permissions and ownership...

Ambari Server 'setup' completed successfully.

将Ambari数据库脚本导入到数据库

#用Ambari用户（上面设置的用户）登录mysql

mysql -u ambari -p

use ambari；

source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

启动Amabri