首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >Ranger同步ldap组问题

Ranger同步ldap组问题

作者头像
soundhearer
发布2020-10-16 11:08:17
2.7K1
发布2020-10-16 11:08:17
举报
文章被收录于专栏:数据湖数据湖

问题描述

按照我们之前的配置,在CDH7.1.1上为Ranger集成OpenLDAP认证这边文章中,我们为Ranger集成了OpenLDAP认证,刚开始给Hive、HDFS、HBase授权的时候,没发现有什么毛病,ldap用户都能正常登录和同步。后来使用Ranger API给用户批量授权时,将大量用户放在同一用户组里,比较好管理。这时我们才发现ldap的用户组没有被Ranger同步过来。

查看openldap,我们可以看到openldap已经创建etl_user用户和用户组。

# etl_user, Group, macro.com
dn: cn=etl_user,ou=Group,dc=macro,dc=com
objectClass: posixGroup
objectClass: top
cn: etl_user
userPassword:: e2NyeXB0fXg=
gidNumber: 50001
memberUid: etl_user

# etl_user, People, macro.com
dn: uid=etl_user,ou=People,dc=macro,dc=com
uid: etl_user
cn: etl_user
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
userPassword:: MTIzNDU2
shadowLastChange: 18235
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash
uidNumber: 50001
gidNumber: 50001
homeDirectory: /home/etl_user

# search result
search: 2
result: 0 Success

# numResponses: 143
# numEntries: 142

通过之前的配置,我们在ranger中也可以看到etl_user用户。

但是我们发现etl_user用户组却没有被同步过来。

问题分析

仔细查看下cloudera官网,发现ranger的ldap用户组配置有误,需要补充如下

修改配置,重启Ranger,发现还是无法同步ldap用户组。这时我们来看一看日志有没有相应的报错

我们可以看到类似的信息:

2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for etl_user
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for oozie
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for hbase
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for sentry
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for impala
2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000

用ldap命令搜索etl_user的用户和用户组,看一下有什么问题:

etl_user用户

[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=People,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <ou=People,dc=macro,dc=com> with scope subtree
# filter: cn=etl_user
# requesting: ALL
#

# etl_user, People, macro.com
dn: uid=etl_user,ou=People,dc=macro,dc=com
uid: etl_user
cn: etl_user
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
userPassword:: MTIzNDU2
shadowLastChange: 18235
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash
uidNumber: 50001
gidNumber: 50001
homeDirectory: /home/etl_user

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

etl_user用户组

[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=Group,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <ou=Group,dc=macro,dc=com> with scope subtree
# filter: cn=etl_user
# requesting: ALL
#

# etl_user, Group, macro.com
dn: cn=etl_user,ou=Group,dc=macro,dc=com
objectClass: posixGroup
objectClass: top
cn: etl_user
userPassword:: e2NyeXB0fXg=
gidNumber: 50001

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

看样子etluser用户组似乎没有拥有etl_user这个用户。我们在etl_user用户组中添加memberUid这个属性

[root@cdh1 ~]# cat etl_user.ldif 
dn: cn=etl_user,ou=Group,dc=macro,dc=com
changetype: modify
add: memberUid
memberUid: etl_user

然后修改用户

[root@cdh1 ~]# ldapmodify -Z -x -W -D "cn=Manager,dc=macro,dc=com" -f etl_user.ldif 
Enter LDAP Password: 
modifying entry "cn=etl_user,ou=Group,dc=macro,dc=com"

再看看etl_user用户组,用户组属性已经被添加进去了。

[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=Group,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <ou=Group,dc=macro,dc=com> with scope subtree
# filter: cn=etl_user
# requesting: ALL
#

# etl_user, Group, macro.com
dn: cn=etl_user,ou=Group,dc=macro,dc=com
objectClass: posixGroup
objectClass: top
cn: etl_user
userPassword:: e2NyeXB0fXg=
gidNumber: 50001
memberUid: etl_user

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

再重启ranger,我们可以发现etl_user用户组已经被同步了

同时,日志的信息已经正确了

2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: longName: etl_user, userName: etl_user
2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No. of members in the group etl_user = 1
2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: LdapDeltaUserGroupBuilder.getGroups() completed with group count: 77
2020-09-22 00:38:18,281 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Unix Auth Service!
2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [SSLv2Hello]
2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1]
2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1.1]
2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1.2]
2020-09-22 00:38:28,755 INFO org.apache.ranger.ldapusersync.process.LdapPolicyMgrUserGroupBuilder: valid cookie saved 
2020-09-22 00:38:29,254 INFO org.apache.ranger.usergroupsync.UserGroupSync: End: initial load of user/group from source==>sink
2020-09-22 00:38:29,254 INFO org.apache.ranger.usergroupsync.UserGroupSync: Done initializing user/group source and sink

小问题

还有个小问题忘了提及,第一次启动Ranger时会出现以下错误,

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes ErrorCode: 1071
2020-09-17 11:04:39,363  [E] ranger_core_db_mysql.sql file import failed!
2020-09-17 11:04:39,364  [I] Unable to create DB schema, Please drop the database and try again

这是因为建立索引的时候,字符长度超过255,由于uef-8字符最长只能为255,超长之后会报错,所以需要对MySQL做如下配置:

set global innodb_file_per_table = on,
innodb_file_format = Barracuda,
innodb_large_prefix = on;
本文参与 腾讯云自媒体分享计划,分享自微信公众号。
原始发表:2020-09-22,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 数据湖 微信公众号,前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体分享计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 问题描述
  • 问题分析
  • 小问题
相关产品与服务
访问管理
访问管理(Cloud Access Management,CAM)可以帮助您安全、便捷地管理对腾讯云服务和资源的访问。您可以使用CAM创建子用户、用户组和角色,并通过策略控制其访问范围。CAM支持用户和角色SSO能力,您可以根据具体管理场景针对性设置企业内用户和腾讯云的互通能力。
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档