0751-7.0.3-如何在CDP DC7.0.3中启用Kerberos

Fayson

发布于 2020-02-24 11:05:16

1.5K0

发布于 2020-02-24 11:05:16

文章被收录于专栏：Hadoop实操

Fayson的github： https://github.com/fayson/cdhproject

文档编写目的

在前面的文章中，Fayson介绍了《0733-7.0.3-如何在Redhat7.6中安装CDP DC7.0.3》，这里我们基于这个环境开始安装Kerberos。Kerberos是一个用于安全认证的第三方协议，并不是Hadoop专用，你可以将其用于其他系统。它采用了传统的共享秘钥方式，实现了在网络环境下不一定保证安全的环境下，Client和Server之间的通信，适用于Client/Server模型，由MIT开发和实现。而使用CDP DC可以较为轻松的实现洁面后的Kerberos集成，本文Fayson主要介绍如何在Readhat7.2的CDP DC7.0.3环境中启用Kerberos。

内容概述

1.如何安装及配置KDC服务

2.如何通过CDP DC启用Kerberos

3.如何验证Kerberos启用成功

4.总结

测试环境

1.操作系统：Redhat7.2

2.CDP DC7.0.3

3.采用root用户操作

KDC服务安装及配置

本文档中将KDC服务安装在Cloudera Manager Server所在服务器上（KDC服务可根据自己需要安装在其他服务器）

1.在Cloudera Manager服务器上安装KDC服务

yum -y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation

2.修改/etc/krb5.conf配置

[root@cdh1 ~]# vim /etc/krb5.conf
[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 default_realm = PREST.COM
# default_ccache_name = KEYRING:persistent:%{uid}

[realms]
 PREST.COM = {
  kdc = cdh1.prest.com
  admin_server = cdh1.prest.com
 }

[domain_realm]
 .prest.com = PREST.COM
 prest.com = PREST.COM

标注部分为需要修改的信息

3.修改/var/kerberos/krb5kdc/kadm5.acl配置

[root@cdh1 ~]# vim /var/kerberos/krb5kdc/kadm5.acl
*/admin@FAYSON.COM      *

4.修改/var/kerberos/krb5kdc/kdc.conf配置

[root@ip-172-31-6-83 ~]# vim /var/kerberos/krb5kdc/kdc.conf
[root@ip-172-31-6-83 ~]# cat /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 PREST.COM = {
  #master_key_type = aes256-cts
  max_renewable_life= 7d 0h 0m 0s
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

标注部分为需要修改的信息。

5.创建Kerberos数据库

[root@cdh1 ~]# kdb5_util create –r PREST.COM -s
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'PREST.COM',
master key name 'K/M@PREST.COM'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key: ******
Re-enter KDC database master key to verify:******

注意：此处需要输入数据库创建数据库的密码。

6.创建Kerberos的管理账号

[root@cdh1 ~]# kadmin.local
Authenticating as principal root/admin@PREST.COM with password.
kadmin.local:  addprinc admin/admin@PREST.COM
WARNING: no policy specified for admin/admin@PREST.COM; defaulting to no policy
Enter password for principal "admin/admin@PREST.COM": ******
Re-enter password for principal "admin/admin@PREST.COM": ******
Principal "admin/admin@PREST.COM" created.
kadmin.local:  exit

注意：在创建账号时需要输入管理员账号及密码。

7.将Kerberos服务添加到自启动服务，并启动krb5kdc和kadmin服务

[root@cdh1 ~]# systemctl enable krb5kdc
Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service.
[root@cdh1 ~]# systemctl enable kadmin
Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service.
[root@cdh1 ~]# systemctl start krb5kdc
[root@cdh1 ~]# systemctl start kadmin

8.测试Kerberos的管理员账号

[root@cdh1 ~]# kinit admin/admin@FAYSON.COM
Password for admin/admin@FAYSON.COM: 
[root@cdh1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin/admin@PREST.COM

Valid starting       Expires              Service principal
09/12/2018 12:54:17  09/13/2018 12:54:17  krbtgt/FAYSON.COM@PREST.COM
        renew until 09/19/2018 12:54:17

9.在集群所有节点执行如下命令，安装Kerberos客户端

yum -y install openldap-clients krb5-workstation krb5-libs

10.将KDC Server上的krb5.conf文件拷贝到所有Kerberos客户端

使用批处理脚本将Kerberos服务端的krb5.conf配置文件拷贝至集群所有节点的/etc目录下：

[root@cdh1 shell]# sh bk_cp.sh node.list /etc/krb5.conf /etc/

至此已完成Kerberos客户端的安装。

集群启用Kerberos

1.在KDC中给Cloudera Manager添加管理员账号

[root@ip-172-31-6-83 shell]# kadmin.local
Authenticating as principal admin/admin@PREST.COM with password.
kadmin.local:   addprinc cloudera-scm/admin@PREST.COM
WARNING: no policy specified for cloudera-scm/admin@PREST.COM; defaulting to no policy
Enter password for principal "cloudera-scm/admin@PREST.COM": ******
Re-enter password for principal "cloudera-scm/admin@PREST.COM": ******
Principal "cloudera-scm/admin@FAYSON.COM" created.
kadmin.local:  exit

验证账号是否可用

2.进入Cloudera Manager的“管理”-> “安全”界面

3.进入如下界面，选择“启用Kerberos”

4.选择KDC服务类型，已经确保KDC服务是否已启动且准备好

5.点击“继续”，配置相关的KDC信息，包括KDC服务器、KDC Realm、加密类型以及待创建的Service Principal的更新生命期等

6.不建议让Cloudera Manager来管理krb5.conf，点击继续

7.输入Cloudera Manager的Kerbers管理员账号，一定得和之前创建的账号一致，点击“继续”

8.点击“继续”，到处KDC Account Manager凭据

9.确认Kerberos信息以及HDFS的端口号的变化（默认即可）

10.点击“继续”，运行启用Kerberos命令

等待集群重启完成

11.点击“继续”

点击“完成”，至此已成功启用Kerberos。

12.查看CM上显示集群已启用Kerberos

集群功能验证

4.1 HDFS功能验证

1.在未认证的情况下用户是无法访问HDFS目录的

2.使用cdhadmin用户进行kinit操作

[root@cdh3 ~]# kinit cdhadmin
[root@cdh3 ~]# klist
[root@cdh3 ~]# hadoop fs -ls /

3.使用hdfs命令put、get以及查看文件

4.2 MR作业验证

[root@cdh3 ~]# hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 5 5

作业运行成功

4.3 Flink作业验证

kinit cdhadmin
hadoop fs -mkdir -p wordcount/input
hadoop fs -rmr wordcount/output
hadoop fs -put dfclear wordcount/input
flink run -m yarn-cluster -yn 3 -yjm 1024 -ytm 1024 /opt/cloudera/parcels/FLINK/lib/flink/examples/streaming/WordCount.jar --input hdfs:///user/cdhadmin/wordcount/input/dfclear --output hdfs:///user/cdhadmin/wordcount/output