0751-7.0.3-如何在CDP DC7.0.3中启用Kerberos
0751-7.0.3-如何在CDP DC7.0.3中启用Kerberos
Fayson的github： https://github.com/fayson/cdhproject
文档编写目的
在前面的文章中，Fayson介绍了《0733-7.0.3-如何在Redhat7.6中安装CDP DC7.0.3》，这里我们基于这个环境开始安装Kerberos。Kerberos是一个用于安全认证的第三方协议，并不是Hadoop专用，你可以将其用于其他系统。它采用了传统的共享秘钥方式，实现了在网络环境下不一定保证安全的环境下，Client和Server之间的通信，适用于Client/Server模型，由MIT开发和实现。而使用CDP DC可以较为轻松的实现洁面后的Kerberos集成，本文Fayson主要介绍如何在Readhat7.2的CDP DC7.0.3环境中启用Kerberos。
- 内容概述
1.如何安装及配置KDC服务
2.如何通过CDP DC启用Kerberos
3.如何验证Kerberos启用成功
4.总结
- 测试环境
1.操作系统：Redhat7.2
2.CDP DC7.0.3
3.采用root用户操作
KDC服务安装及配置
本文档中将KDC服务安装在Cloudera Manager Server所在服务器上（KDC服务可根据自己需要安装在其他服务器）
1.在Cloudera Manager服务器上安装KDC服务
yum -y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation
2.修改/etc/krb5.conf配置
[root@cdh1 ~]# vim /etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
dns_lookup_realm = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
rdns = false
default_realm = PREST.COM
# default_ccache_name = KEYRING:persistent:%{uid}
[realms]
PREST.COM = {
kdc = cdh1.prest.com
admin_server = cdh1.prest.com
}
[domain_realm]
.prest.com = PREST.COM
prest.com = PREST.COM
标注部分为需要修改的信息
3.修改/var/kerberos/krb5kdc/kadm5.acl配置
[root@cdh1 ~]# vim /var/kerberos/krb5kdc/kadm5.acl
*/admin@FAYSON.COM *
4.修改/var/kerberos/krb5kdc/kdc.conf配置
[root@ip-172-31-6-83 ~]# vim /var/kerberos/krb5kdc/kdc.conf
[root@ip-172-31-6-83 ~]# cat /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
kdc_ports = 88
kdc_tcp_ports = 88
[realms]
PREST.COM = {
#master_key_type = aes256-cts
max_renewable_life= 7d 0h 0m 0s
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
}
标注部分为需要修改的信息。
5.创建Kerberos数据库
[root@cdh1 ~]# kdb5_util create –r PREST.COM -s
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'PREST.COM',
master key name 'K/M@PREST.COM'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key: ******
Re-enter KDC database master key to verify:******
注意：此处需要输入数据库创建数据库的密码。
6.创建Kerberos的管理账号
[root@cdh1 ~]# kadmin.local
Authenticating as principal root/admin@PREST.COM with password.
kadmin.local: addprinc admin/admin@PREST.COM
WARNING: no policy specified for admin/admin@PREST.COM; defaulting to no policy
Enter password for principal "admin/admin@PREST.COM": ******
Re-enter password for principal "admin/admin@PREST.COM": ******
Principal "admin/admin@PREST.COM" created.
kadmin.local: exit
注意：在创建账号时需要输入管理员账号及密码。
7.将Kerberos服务添加到自启动服务，并启动krb5kdc和kadmin服务
[root@cdh1 ~]# systemctl enable krb5kdc
Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service.
[root@cdh1 ~]# systemctl enable kadmin
Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service.
[root@cdh1 ~]# systemctl start krb5kdc
[root@cdh1 ~]# systemctl start kadmin
8.测试Kerberos的管理员账号
[root@cdh1 ~]# kinit admin/admin@FAYSON.COM
Password for admin/admin@FAYSON.COM:
[root@cdh1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin/admin@PREST.COM
Valid starting Expires Service principal
09/12/2018 12:54:17 09/13/2018 12:54:17 krbtgt/FAYSON.COM@PREST.COM
renew until 09/19/2018 12:54:17
9.在集群所有节点执行如下命令，安装Kerberos客户端
yum -y install openldap-clients krb5-workstation krb5-libs
10.将KDC Server上的krb5.conf文件拷贝到所有Kerberos客户端
使用批处理脚本将Kerberos服务端的krb5.conf配置文件拷贝至集群所有节点的/etc目录下：
[root@cdh1 shell]# sh bk_cp.sh node.list /etc/krb5.conf /etc/
至此已完成Kerberos客户端的安装。
集群启用Kerberos
1.在KDC中给Cloudera Manager添加管理员账号
[root@ip-172-31-6-83 shell]# kadmin.local
Authenticating as principal admin/admin@PREST.COM with password.
kadmin.local: addprinc cloudera-scm/admin@PREST.COM
WARNING: no policy specified for cloudera-scm/admin@PREST.COM; defaulting to no policy
Enter password for principal "cloudera-scm/admin@PREST.COM": ******
Re-enter password for principal "cloudera-scm/admin@PREST.COM": ******
Principal "cloudera-scm/admin@FAYSON.COM" created.
kadmin.local: exit
验证账号是否可用
2.进入Cloudera Manager的“管理”-> “安全”界面
3.进入如下界面，选择“启用Kerberos”
4.选择KDC服务类型，已经确保KDC服务是否已启动且准备好
5.点击“继续”，配置相关的KDC信息，包括KDC服务器、KDC Realm、加密类型以及待创建的Service Principal的更新生命期等
6.不建议让Cloudera Manager来管理krb5.conf，点击继续
7.输入Cloudera Manager的Kerbers管理员账号，一定得和之前创建的账号一致，点击“继续”
8.点击“继续”，到处KDC Account Manager凭据
9.确认Kerberos信息以及HDFS的端口号的变化（默认即可）
10.点击“继续”，运行启用Kerberos命令
等待集群重启完成
11.点击“继续”
点击“完成”，至此已成功启用Kerberos。
12.查看CM上显示集群已启用Kerberos
集群功能验证
4.1 HDFS功能验证
1.在未认证的情况下用户是无法访问HDFS目录的
2.使用cdhadmin用户进行kinit操作
[root@cdh3 ~]# kinit cdhadmin
[root@cdh3 ~]# klist
[root@cdh3 ~]# hadoop fs -ls /
3.使用hdfs命令put、get以及查看文件
4.2 MR作业验证
[root@cdh3 ~]# hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 5 5
作业运行成功
4.3 Flink作业验证
kinit cdhadmin
hadoop fs -mkdir -p wordcount/input
hadoop fs -rmr wordcount/output
hadoop fs -put dfclear wordcount/input
flink run -m yarn-cluster -yn 3 -yjm 1024 -ytm 1024 /opt/cloudera/parcels/FLINK/lib/flink/examples/streaming/WordCount.jar --input hdfs:///user/cdhadmin/wordcount/input/dfclear --output hdfs:///user/cdhadmin/wordcount/output
作业执行成功
Yarn上记录的信息
Flink的History显示
总结
1.CDP DC集群的Kerberos启用与CDH5和CDH6差别不大，只是在界面上有小的改动
2.CDP DC的KDC类型支持FreeIPA服务
3.在CDH集群中启用Kerberos需要先安装Kerberos服务（krb5kdc和kadmin服务）
4.在集群所有节点需要安装Kerberos客户端，用于和kdc服务通信
5.在Cloudera Manager Server节点需要额外安装openldap-clients包