Ambari安装与集群配置

参考网址： http://www.ibm.com/developerworks/cn/opensource/os-cn-bigdata-ambari/index.html http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/index.html

第一部分：基本环境准备

    准备3个节点，下面操作需要在每个节点上执行。     参考网址http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/prepare_the_environment.html 1、配置IP和hostname     [root@node1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eno16777736     [root@node1 ~]# vi /etc/hosts     注意：不要从您的hosts文件中删除以下两行。 删除或编辑以下行可能会导致需要网络功能的各种程序失败。     127.0.0.1 localhost.localdomain localhost     ::1 localhost6.localdomain6 localhost6     [root@node1 ~]# vi /etc/hostname     [root@node1 ~]# service network restart 2、更改yum源 2.1 下载国内镜像源repo文件     如果已经安装wget工具，则可以通过下面命令下，自动载配置     [root@node1 ~]# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo     --2017-01-02 21:44:57--  http://mirrors.aliyun.com/repo/Centos-7.repo     Resolving mirrors.aliyun.com (mirrors.aliyun.com)... 115.28.122.210, 112.124.140.210     Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|115.28.122.210|:80... connected.     HTTP request sent, awaiting response... 200 OK     Length: 2573 (2.5K) [application/octet-stream]     Saving to: ‘/etc/yum.repos.d/CentOS-Base.repo’     100%[========================================================================>] 2,573       --.-K/s   in 0s           2017-01-02 21:44:58 (231 MB/s) - ‘/etc/yum.repos.d/CentOS-Base.repo’ saved [2573/2573]     如果没有安装wget工具，直接下载http://mirrors.aliyun.com/repo/Centos-7.repo     然后将下载的阿里镜像源文件移到/etc/yum.repos.d下即可（重命名CentOS-Base.repo）     [root@node1 ~]# mv Centos-7.repo /etc/yum.repos.d/CentOS-Base.repo 2.2 更新yum缓存     [root@node1 ~]# yum clean all     Loaded plugins: fastestmirror     Cleaning repos: base c7-media extras updates     Cleaning up everything     Cleaning up list of fastest mirrors     [root@node1 ~]# yum makecache     ......     Determining fastest mirrors      * base: mirrors.aliyun.com      * extras: mirrors.aliyun.com      * updates: mirrors.aliyun.com     Metadata Cache Created 3、关闭防火墙和SELINUX     [root@node1 ~]# systemctl disable firewalld      Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.      Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.     [root@node1 ~]# service firewalld stop      Redirecting to /bin/systemctl stop  firewalld.service     [root@node1 ~]# setenforce 0     [root@node1 ~]# vi /etc/selinux/config      SELINUX=disabled     设置umask值     [root@node1 ~]echo umask 0022 >> /etc/profile 4、设置SSH无密钥登录

4.1 每个节点生成无密钥SSH

[root@node1 ~]#yum install openssh-server

    [root@node1 ~]# ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa     [root@node1 ~]# cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys     [root@node1 ~]# chmod 0600 ~/.ssh/authorized_keys

4.2    相互访问     [root@anode1 ~]#vi /etc/hosts     127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4     ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6     172.16.54.201 anode1     172.16.54.202 anode2     172.16.54.203 anode3     [root@anode1 ~]#scp /etc/hosts 172.16.54.202:/etc/

    [root@anode1 ~]#scp /etc/hosts 172.16.54.203:/etc/

  然后在每个节点执行sh sshKeyGen.sh

5、设置时间同步     [root@node1 ~]# yum install -y ntp     [root@node1 ~]# systemctl enable ntpd      Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.     [root@node1 ~]# systemctl start ntpd 6、安装配置JDK 1.8 6.1 卸载预装的JDK     rpm -qa|grep jdk     yum -y remove java java-1.7.0-openjdk-1.7.0.79-2.5.5.4.el6.x86_64     rpm -qa|grep java     yum -y remove java tzdata-java-2015e-1.el6.noarch 6.2 配置新的JDK     注意，每个节点的jdk安装到相同的目录下面，比如都安装到/opt目录下。     [root@node1 ~]#vi /etc/profile     #set Java Path     export JAVA_HOME=/opt/java/jdk1.8.0_101     export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib     export PATH=$PATH:$JAVA_HOME/bin     [root@node1 ~]# source /etc/profile     [root@node1 ~]# java -version 7、拾遗

7.1 设置buildNumber

    这一部分是针对于ambari源码编译出现buildNumber缺失一种解决办法。

    [root@node1 ~]# vi .bash_profile     export buildNumber=9999     [root@anode1 ~]# source .bash_profile     [root@node1 ~]# echo $buildNumber     9999 7.2 确认是否安装了wget     [root@anode1 ~]# yum install -y wget 第二部分 Ambari-server安装与配置 1、下载Ambari 的公共库文件（public repository）     http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari.repo     [root@anode2 ~]# wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari.repo -O /etc/yum.repos.d/ambari.repo     2017-01-03 06:41:08 URL:http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari.repo [287/287] -> "/etc/yum.repos.d/ambari.repo" [1] 2、修改ambari.repo文件     如果访问hortonworks.com速度顺畅，则不要修改ambari.repo，可以跳过这一步。     当访问hortonworks.com速度较慢时，可以建立本地repository。     [root@vnode0 yum.repos.d]# cat ambari.repo     #VERSION_NUMBER=2.4.2.0-136     [Updates-ambari-2.4.2.0]     name=ambari-2.4.2.0 - Updates     baseurl=http://192.168.1.180/ambari/centos7/2.4.2.0-136/     gpgcheck=0     gpgkey=http://192.168.1.180/ambari/centos7/2.4.2.0-136//RPM-GPG-KEY/RPM-GPG-KEY-Jenkins     enabled=1

    priority=1

    然后将ambari.repo复制到其他节点     [root@node1 yum.repos.d]# scp ambari.repo 172.16.54.202:/etc/yum.repos.d/

3、安装Ambari-server     [root@east81 yum.repos.d]# yum install ambari-server     Loaded plugins: fastestmirror, langpacks     Loading mirror speeds from cached hostfile     Resolving Dependencies     ==================================================================================================================      Package                                    Arch                            Version                                    Repository                                       Size    =====================================================================================================================     Installing:      ambari-server                              x86_64                          2.4.2.0-136                                Updates-ambari-2.4.2.0                          645 M     Installing for dependencies:      postgresql                                 x86_64                          9.2.13-1.el7_1                             os                                              3.0 M      postgresql-libs                            x86_64                          9.2.13-1.el7_1                             os                                              230 k      postgresql-server                          x86_64                          9.2.13-1.el7_1                             os                                              3.8 M     Transaction Summary     ======================================================================================================================     Install  1 Package (+3 Dependent packages)     Total download size: 652 M     Installed size: 732 M     Is this ok [y/d/N]: y     Downloading packages:     (1/4): postgresql-libs-9.2.13-1.el7_1.x86_64.rpm                                                                                                      | 230 kB  00:00:00          (2/4): postgresql-9.2.13-1.el7_1.x86_64.rpm                                                                                                           | 3.0 MB  00:00:00          (3/4): postgresql-server-9.2.13-1.el7_1.x86_64.rpm                                                                                                    | 3.8 MB  00:00:00          ambari-server-2.4.2.0-136.x86_ FAILED                                                                                                     ]  159 B/s | 7.8 MB 1180:43:26 ETA     http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari/ambari-server-2.4.2.0-136.x86_64.rpm: [Errno 12] Timeout on http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari/ambari-server-2.4.2.0-136.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')     Trying other mirror.     ambari-server-2.4.2.0-136.x86_ FAILED                                          -                                                           ]  437 B/s |  31 MB 414:04:07 ETA     http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari/ambari-server-2.4.2.0-136.x86_64.rpm: [Errno 12] Timeout on http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari/ambari-server-2.4.2.0-136.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')     Trying other mirror.     warning: /var/cache/yum/x86_64/7/Updates-ambari-2.4.2.0/packages/ambari-server-2.4.2.0-136.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY MB  00:00:00 ETA     Public key for ambari-server-2.4.2.0-136.x86_64.rpm is not installed     (4/4): ambari-server-2.4.2.0-136.x86_64.rpm                                                                                                           | 645 MB  00:17:12          -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------     Total                                                                                                                                        295 kB/s | 652 MB  00:37:44          Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins     Importing GPG key 0x07513CAD:      Userid     : "Jenkins (HDP Builds) <jenkin@hortonworks.com>"      Fingerprint: df52 ed4f 7a3a 5882 c099 4c66 b973 3a7a 0751 3cad      From       : http://public-repo-1.hortonworks.com/ambari/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins     Is this ok [y/N]: y     Running transaction check     Running transaction test     Transaction test succeeded     Running transaction       Installing : postgresql-libs-9.2.13-1.el7_1.x86_64                                                                                                           1/4       Installing : postgresql-9.2.13-1.el7_1.x86_64                                                                                                                2/4       Installing : postgresql-server-9.2.13-1.el7_1.x86_64                                                                                                         3/4       Installing : ambari-server-2.4.2.0-136.x86_64                                                                                                                4/4       Verifying  : postgresql-9.2.13-1.el7_1.x86_64                                                                                                                1/4       Verifying  : ambari-server-2.4.2.0-136.x86_64                                                                                                                2/4       Verifying  : postgresql-libs-9.2.13-1.el7_1.x86_64                                                                                                           3/4       Verifying  : postgresql-server-9.2.13-1.el7_1.x86_64                                                                                                         4/4     Installed:       ambari-server.x86_64 0:2.4.2.0-136                                                                                                                                              Dependency Installed:       postgresql.x86_64 0:9.2.13-1.el7_1              postgresql-libs.x86_64 0:9.2.13-1.el7_1               postgresql-server.x86_64 0:9.2.13-1.el7_1                       Complete!     [root@east81 yum.repos.d]# 4、设置Ambari-server     [root@anode1 opt]# ambari-server setup     Using python  /usr/bin/python     Setup ambari-server     Checking SELinux...     SELinux status is 'disabled'     Customize user account for ambari-server daemon [y/n] (n)? y    #输入y     Enter user account for ambari-server daemon (root):             #回车，默认值     Adjusting ambari-server permissions and ownership...     Checking firewall status...     Checking JDK...     [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8     [2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7     [3] Custom JDK     ==============================================================================     Enter choice (1): 3                                             #输入3     WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.     WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.     Path to JAVA_HOME: /opt/jdk1.8.0_101                            #输入JDK根目录     Validating JDK on Ambari Server...done.     Completing setup...     Configuring database...     Enter advanced database configuration [y/n] (n)?                #回车，默认PostgreSQL     Configuring database...     Default properties detected. Using built-in database.     Configuring ambari database...     Checking PostgreSQL...     Running initdb: This may take up to a minute.     Initializing database ... OK     About to start PostgreSQL     Configuring local database...     Connecting to local database...done.     Configuring PostgreSQL...     Restarting PostgreSQL     Extracting system views...     ambari-admin-2.4.2.0.0.jar     Adjusting ambari-server permissions and ownership...     Ambari Server 'setup' completed successfully.     [root@anode1 opt]# 5、启动Ambari-server     [root@anode1 opt]# ambari-server start     Using python  /usr/bin/python     Starting ambari-server     Ambari Server running with administrator privileges.     Organizing resource files at /var/lib/ambari-server/resources...     Ambari database consistency check started...     No errors were found.     Ambari database consistency check finished     Server PID at: /var/run/ambari-server/ambari-server.pid     Server out at: /var/log/ambari-server/ambari-server.out     Server log at: /var/log/ambari-server/ambari-server.log     Waiting for server start....................     Ambari Server 'start' completed successfully. 第三部分 集群配置 1、登录Ambari-server     比如，Ambari-server所在节点node1的IP是172.16.54.201     在浏览器中打开http://172.16.54.201:8080     用户名和密码都是admin,即可登录Ambari-server 2、单击“Launch Install Wizard”按钮，启动Ambari安装向导     (1) 在“Name your cluster”输入框输入集群名称，比如cc     (2) Select Version选择版本,选择HDP2.4         如果“使用公共存储库”，保留“redhat7”即可，其他系统删除。         如果“使用本地存储库”，同样保留“redhat7”即可，删除其他系统。修改：         HDP-2.4输入：http://192.168.1.180/hdp/HDP/centos7/2.x/updates/2.4.3.0/

        HDP-UTILS-1.1.0.20输入：http://192.168.1.180/hdp/HDP-UTILS-1.1.0.20/repos/centos7/

提示：会在各个节点生成hdp repo文件

       [root@a1 yum.repos.d]# ls        ambari.repo  bak  CentOS-Base.repo  HDP.repo  HDP-UTILS.repo 

       [root@a1 yum.repos.d]# more HDP.repo        [HDP-2.4]        name=HDP-2.4        baseurl=http://192.168.1.60/hdp/HDP/centos7/2.x/updates/2.4.3.0/        path=/        enabled=1        gpgcheck=0        [root@a1 yum.repos.d]# more HDP-UTILS.repo        [HDP-UTILS-1.1.0.20]        name=HDP-UTILS-1.1.0.20        baseurl=http://192.168.1.60/hdp/HDP-UTILS-1.1.0.20/repos/centos7/        path=/        enabled=1        gpgcheck=0

    (3)Install Options安装选项         在Target Hosts：输入节点的hostname值，不是IP。比如：         anode1         anode2         anode3     (4) Host Registration Information主机注册信息         将Ambari-server主机的SSH私钥复制到输入框         [root@anode1 ~]# cat /root/.ssh/id_rsa     (5) 确认主机，进行anode1、anode2、anode3节点注册     (6)    选择服务,先选择最小服务安装,后续再安装其他服务。         建议选择ZooKeeper和Ambari Metrics     (7)设置Grafana Admin Password         输入123456即可     (8)分配从属机和客户端,所有节点选择安装Client 第四部分：遇到问题 1、警告 Ambari Metrics     hbase_master_heapsize     512          Value is less than the recommended default of 640     HBase Master Heap Size. In embedded mode, total heap size is sum of master and regionserver heap sizes.     解决办法：     Ambari Metrics配置，检索hbase_master_heapsize，然后将值修改为640，单击保存按钮     最后重启组件。

 2、ambari 报Heartbeat Lost问题

     重启ambari-agent 

     [root@anode1 ~]# ambari-server start