https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPMS/x86_64/
CDH6.2.0安装包地址:
https://archive.cloudera.com/cdh6/6.2.0/parcels/
由于我们的操作系统为CentOS7,需要下载以下文件:
当然你还可以选择我的资料
链接:
https://pan.baidu.com/s/1dj-UhMKkxsj7Aw9btqcuZg
提取码:zo7r
系统准备
1. 准备虚拟机(根据自己的系统资源分配虚拟机资源)
操作系统 | CPU | 内存 | 硬盘 | hostname | ip | 角色 |
---|---|---|---|---|---|---|
CentOS(64位) | 2vCPU | 16GiB | 800GiB | manager | 192.168.124.100 | clouder-manager |
CentOS(64位) | 2vCPU | 32GiB | 800GiB | node1 | 192.168.124.101 | |
CentOS(64位) | 2vCPU | 32GiB | 800GiB | node2 | 192.168.124.102 | |
CentOS(64位) | 2vCPU | 32GiB | 800GiB | node3 | 192.168.124.103 |
2. 静态IP设置(每个节点)
sudo su -- 切换到root用户
vi /etc/sysconfig/network-scripts/ifcfg-ens192
service network restart 重启网络生效
yum install -y net-tools ifconfig查看设置
3. 编辑/etc/hosts文件(每个节点)
vim /etc/hosts
[root@10-43-75-89 ecarx_dev]# hostnamectl set-hostname slave02
192.168.124.100 master
192.168.124.101 slave01
192.168.124.102 slave02
192.168.124.103 slave03
4. 关闭防火墙、禁止防火墙开机自启(每个节点)
5. ssh无密码登录,只需要执行红色的都行
[root@master parcel]# for i in {1..2}; do ssh -p52222 root@slave0$i cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys; done |
---|
[root@master parcel]# for i in {1..2}; do scp -P52222 /root/.ssh/authorized_keys root@slave0$i:/root/.ssh/authorized_keys; done |
---|
ssh-keygen -t rsa |
---|
[root@master ecarx_dev]# ssh-copy-id master [root@master ecarx_dev]# ssh-copy-id slave01 [root@master ecarx_dev]# ssh-copy-id slave02 |
---|
scp -P52222 ./aa.txt 10.43.75.89:/ |
---|
1、scp指定端口传输,端口需放在scp后面 scp -P52222 -r ~/.ssh/ root@10.43.16.89:~/.ssh/ scp -P 34543 -r spark xiaojp@120.26.233.3:~/2、ssh指定端口登录:ssh -p 34543 xiaojp@120.26.233.3 scp -P52222 ./abc.txt root@10.43.16.89:/root |
---|
6. 配置NTP服务(所有节点如果公司的时间同步则可以跳过)
7. 修改Linux swappiness参数(所有节点)
为了避免服务器使用swap功能而影响服务器性能,一般都会把vm.swappiness修改为0(cloudera建议10以下)
echo 0 > /proc/sys/vm/swappiness
这样操作重启机器还是还原
修改配置文件,开机自动设置参数
编辑vim /etc/sysctl.conf
在最后添加vm.swappiness=0
修改完确认:
重启不生效:
https://blog.csdn.net/ygtlovezf/article/details/79014299
8. 禁用透明页(所有节点)
9. JDK安装(所有节点)
# java
export JAVA_HOME=/usr/java/jdk1.8.0_181-clouderaexport
CLASSPATH=.:$CLASSPTAH:$JAVA_HOME/libexport
PATH=$PATH:$JAVA_HOME/bin
10. 创建/usr/share/java目录,将mysql-jdbc包放过去(所有节点)
11. 为保证防火墙、虚拟机参数修改后生效,各节点机器需要重启 reboot
12. Mysql安装
$>mysql -u root
mysql>use mysql;
mysql>update user set authentication_string = password('123456'), password_expired = 'N', password_last_changed = now() where user = 'root';
mysql>exit;
在这里进行之后,把skip-grant-tables 注释,重新启动mysql,链接mysql
mysql -uroot -p123456
CREATE DATABASE test DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
CREATE DATABASE cmserver DEFAULT CHARSET utf8 COLLATE utf8_general_ci; GRANT ALL ON cmserver.* TO 'cmserver'@'%' IDENTIFIED BY 'Yyf5211314!'; hive中的元数据 CREATE DATABASE metastore DEFAULT CHARSET utf8 COLLATE utf8_general_ci; GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'Yyf5211314!'; Activity Monitor CREATE DATABASE amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci; GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'Yyf5211314!'; ReportsManager CREATE DATABASE rman DEFAULT CHARSET utf8 COLLATE utf8_general_ci; GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'Yyf5211314!'; CREATE DATABASE oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci; GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'Yyf5211314!'; HUE CREATE DATABASE hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci; GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'Yyf5211314!'; Navigator Audit Server CREATE DATABASE nav DEFAULT CHARSET utf8 COLLATE utf8_general_ci; GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'Yyf5211314!'; Navigator Metadata Server CREATE DATABASE navms DEFAULT CHARSET utf8 COLLATE utf8_general_ci; GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'Yyf5211314!'; |
---|
授权用户 (1)任意主机以用户root和密码pwd连接到mysql服务器 mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'pwd' WITH GRANT OPTION; mysql> flush privileges; (2)指定IP为(如192.168.1.100)的主机以用户tuser和密码tpwd连接到mysql服务器 mysql> GRANT ALL PRIVILEGES ON *.* TO 'tuser'@'192.168.1.100' IDENTIFIED BY 'tpwd' WITH GRANT OPTION; mysql> flush privileges; |
---|
//允许root用户进行访问链接
use mysql
mysql> select host, user from user; +-----------+---------------+ | host | user | +-----------+---------------+ | % | amon | | % | amonuser | | % | cmserver | | % | cmserveruser | | % | hive | | % | hiveuser | | % | hue | | % | hueuser | | % | nav | | % | navms | | % | oozie | | % | oozieuser | | % | rman | | % | rmanuser | | % | root | | % | sentry | | localhost | mysql.session | | localhost | mysql.sys | | localhost | root | +-----------+---------------+ |
---|
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%'IDENTIFIED BY '123456' WITH GRANT OPTION; FLUSH PRIVILEGES; |
---|
13. 安装Httpd服务(manager)
14. 防止出现莫名错误,事先安装以下插件(所有节点)(我只在主节点)这一步我没成功,而是更新本地的yum源以后,进行的第17没有执行
15. 配置Cloudera Manager包,配置本地yum源(manager节点)
[cloudera-manager] name = Cloudera Manager, Version baseurl = http://master/cloudera-repos/cm6/ gpgcheck = 1 |
---|
注意路径:http://master/cloudera-repos/cm6/
[root@master cm5.16]# yum clean all && yum makecache Loaded plugins: fastestmirror Cleaning repos: base cloudera-manager epel extras updates Cleaning up everything Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos Cleaning up list of fastest mirrors Loaded plugins: fastestmirror base | 2.9 kB 00:00:00 cloudera-manager | 2.9 kB 00:00:00 epel | 2.9 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 (1/15): base/primary_db | 6.0 MB 00:00:00 (2/15): base/other_db | 2.6 MB 00:00:00 (3/15): cloudera-manager/filelists_db | 88 kB 00:00:00 (4/15): cloudera-manager/primary_db | 11 kB 00:00:00 (5/15): base/filelists_db | 7.1 MB 00:00:00 (6/15): cloudera-manager/other_db | 1.1 kB 00:00:00 (7/15): epel/filelists_db | 14 MB 00:00:00 (8/15): epel/primary_db | 7.4 MB 00:00:00 (9/15): epel/other_db | 3.5 MB 00:00:00 (10/15): extras/filelists_db | 250 kB 00:00:00 (11/15): extras/primary_db | 216 kB 00:00:00 (12/15): extras/other_db | 132 kB 00:00:00 (13/15): updates/filelists_db | 5.2 MB 00:00:00 (14/15): updates/other_db | 764 kB 00:00:00 (15/15): updates/primary_db | 7.4 MB 00:00:00 Determining fastest mirrors Metadata Cache Created [root@master cm5.16]# |
---|
16.导入GPG key(如果没有这步操作,很可能cloudera服务安装失败)manager节点
rpm--import https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPM-GPG-KEY-cloudera
17. 安装 Cloudera Manager(manager节点 和最上面类似14)