专栏首页数据库干货铺大数据安全利器ranger 编译安装

大数据安全利器ranger 编译安装

ranger大数据领域的一个集中式安全管理框架,它可以对诸如hdfs、hive、kafka、storm等组件进行细粒度的权限控制。本文将介绍部署过程

1. 部署准备

ranger: 进入apach官网下载 http://ranger.apache.org/download.html, 本次使用的是ranger1.2.0 ,地址为http://mirror.bit.edu.cn/apache/ranger/1.2.0/apache-ranger-1.2.0.tar.gz

maven: 进入Apache的maven官网http://maven.apache.org/download.cgi下载, 本次用的是maven3.6 ,下载地址 http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz

python2.7: 因编译及试用中需要Python2.7版本的Python,因此如果为Centos6系统,需要手动升级Python至Python2.7,升级过程可参考历史文章Python升级

MySQL: 需要mysql数据库,如无可用MySQL需要部署一套MySQL,部署方法请参考历史文章MySQL部署

mysql-connector-java: 进入MySQL官网下载 https://dev.mysql.com/downloads/connector/j/5.1.html

bc命令: 使用过程中需要使用bc命令

## 下载ranger
wget  http://mirror.bit.edu.cn/apache/ranger/1.2.0/apache-ranger-1.2.0.tar.gz
##  下载maven
wget  http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz
##  安装bc命令
yum  install bc -y

2. 编译安装过程

2.1 解压

#  解压ranger及maven包
tar  -zxvf apache-ranger-1.2.0.tar.gz
tar -zxvf  apache-maven-3.6.1-bin.tar.gz

#  将maven相关命令路径配置到环境变量
vim  /etc/profile
export PATH=/usr/local/maven/bin:$PATH

2.2 编译

cd apache-ranger-1.2.0
mvn clean
# 编译
mvn clean compile package assembly:assembly install -DskipTests -Drat.skip=true
编译过程中会有如下信息

执行过程中会出现很多信息,且编译时间很长,如果出现异常,根据对应的报错信息进行处理,例如Python包不存在时需要手动添加;如果某个jar不存在,可以手动下载放置对应的路径或修改pom地址或文件。

经过很长一段时间,且多次解决错误后,编译完成,出现如下结果

[INFO] Reactor Summary for ranger 1.2.0:
[INFO] 
[INFO] ranger ............................................. SUCCESS [  0.244 s]
[INFO] Jdbc SQL Connector ................................. SUCCESS [  0.573 s]
[INFO] Credential Support ................................. SUCCESS [  0.391 s]
[INFO] Audit Component .................................... SUCCESS [  1.065 s]
[INFO] Common library for Plugins ......................... SUCCESS [  1.332 s]
[INFO] Installer Support Component ........................ SUCCESS [  0.153 s]
[INFO] Credential Builder ................................. SUCCESS [  0.293 s]
[INFO] Embedded Web Server Invoker ........................ SUCCESS [  0.345 s]
[INFO] Key Management Service ............................. SUCCESS [  0.984 s]
[INFO] ranger-plugin-classloader .......................... SUCCESS [  0.209 s]
[INFO] HBase Security Plugin Shim ......................... SUCCESS [  0.830 s]
[INFO] HBase Security Plugin .............................. SUCCESS [  1.092 s]
[INFO] Hdfs Security Plugin ............................... SUCCESS [  0.786 s]
[INFO] Hive Security Plugin ............................... SUCCESS [  1.135 s]
[INFO] Knox Security Plugin Shim .......................... SUCCESS [  0.524 s]
[INFO] Knox Security Plugin ............................... SUCCESS [  0.712 s]
[INFO] Storm Security Plugin .............................. SUCCESS [  0.533 s]
[INFO] YARN Security Plugin ............................... SUCCESS [  0.398 s]
[INFO] Ranger Util ........................................ SUCCESS [  1.099 s]
[INFO] Unix Authentication Client ......................... SUCCESS [  0.259 s]
[INFO] Security Admin Web Application ..................... SUCCESS [ 12.847 s]
[INFO] KAFKA Security Plugin .............................. SUCCESS [  0.458 s]
[INFO] SOLR Security Plugin ............................... SUCCESS [  1.208 s]
[INFO] NiFi Security Plugin ............................... SUCCESS [  0.386 s]
[INFO] NiFi Registry Security Plugin ...................... SUCCESS [  0.381 s]
[INFO] Unix User Group Synchronizer ....................... SUCCESS [  1.016 s]
[INFO] Ldap Config Check Tool ............................. SUCCESS [  0.222 s]
[INFO] Unix Authentication Service ........................ SUCCESS [  0.330 s]
[INFO] KMS Security Plugin ................................ SUCCESS [  0.490 s]
[INFO] Tag Synchronizer ................................... SUCCESS [  1.211 s]
[INFO] Hdfs Security Plugin Shim .......................... SUCCESS [  0.311 s]
[INFO] Hive Security Plugin Shim .......................... SUCCESS [  0.800 s]
[INFO] YARN Security Plugin Shim .......................... SUCCESS [  0.320 s]
[INFO] Storm Security Plugin shim ......................... SUCCESS [  0.365 s]
[INFO] KAFKA Security Plugin Shim ......................... SUCCESS [  0.313 s]
[INFO] SOLR Security Plugin Shim .......................... SUCCESS [  0.701 s]
[INFO] Atlas Security Plugin Shim ......................... SUCCESS [  0.398 s]
[INFO] KMS Security Plugin Shim ........................... SUCCESS [  0.342 s]
[INFO] ranger-examples .................................... SUCCESS [  0.042 s]
[INFO] Ranger Examples - Conditions and ContextEnrichers .. SUCCESS [  0.335 s]
[INFO] Ranger Examples - SampleApp ........................ SUCCESS [  0.140 s]
[INFO] Ranger Examples - Ranger Plugin for SampleApp ...... SUCCESS [  0.322 s]
[INFO] Ranger Tools ....................................... SUCCESS [  0.457 s]
[INFO] Atlas Security Plugin .............................. SUCCESS [  0.722 s]
[INFO] Sqoop Security Plugin .............................. SUCCESS [  0.472 s]
[INFO] Sqoop Security Plugin Shim ......................... SUCCESS [  0.293 s]
[INFO] Kylin Security Plugin .............................. SUCCESS [  0.410 s]
[INFO] Kylin Security Plugin Shim ......................... SUCCESS [  0.315 s]
[INFO] Unix Native Authenticator .......................... SUCCESS [  0.491 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:24 h
[INFO] Finished at: 2019-08-19T09:47:05+08:00
[INFO] ------------------------------------------------------------------------

在target文件夹下将有如下包

即表示编译成功完成。

2.3 安装并启动ranger-admin

2.3.1 修改配置文件

关于数据库安装,权限设置等,本文不再展开。

#  进入target目录
cd /opt/apache-ranger-1.2.0/target/

#  解压ranger-1.2.0-admin.tar.gz
tar  -zxvf  ranger-1.2.0-admin.tar.gz

# 进入ranger-1.2.0-admin目录
cd  ranger-1.2.0-admin

# 修改 install.properties
vim install.properties


SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java-8.0.17.jar   //  修改为准备工作中下载的jar包及路径

db_root_user=root
db_root_password=123456
db_host=192.168.56.105

db_name=ranger
db_user=rangeradmin
db_password=rangeradmin


# 可以注销如下内容
#Source for Audit Store. Currently only solr is supported.
# * audit_store is solr
## audit_store=solr

# * audit_solr_url URL to Solr. E.g. http://<solr_host>:6083/solr/ranger_audits
## audit_solr_urls=
## audit_solr_user=
## audit_solr_password=
## audit_solr_zookeepers=

2.3.2 修改 setup.sh

// 注销如下内容

## if [ "$?" != "0" ]
## then
##         validateDefaultUsersPassword 'admin' "${rangerAdmin_password}"
##         validateDefaultUsersPassword 'rangertagsync' "${rangerTagsync_password}"
##         validateDefaultUsersPassword 'rangerusersync' "${rangerUsersync_password}"
##         validateDefaultUsersPassword 'keyadmin' "${keyadmin_password}"
## fi

2.3.3 安装

./ setup.sh

注意,如果中途报如下错误

2019-08-20 08:54:22,460  [I] '/usr/local/java/bin/java' command found
setup.sh:行325: bc: 未找到命令
setup.sh: 第 325 行:[: -eq: 期待一元表达式

即缺少bc命令,安装后即可

最终出现如下结果

2019-08-20 09:00:18,240  [I] --------- Verifying Ranger DB connection ---------
2019-08-20 09:00:18,240  [I] Checking connection..
geradmin' -p '********' -noheader -trim -c \; -query "SELECT version();"
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2019-08-20 09:00:18,838  [I] Checking connection passed.
2019-08-20 09:00:19,091  [I] DB FLAVOR :MYSQL
2019-08-20 09:00:19,092  [I] --------- Verifying Ranger DB connection ---------
2019-08-20 09:00:19,092  [I] Checking connection..
geradmin' -p '********' -noheader -trim -c \; -query "SELECT version();"
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2019-08-20 09:00:19,660  [I] Checking connection passed.
Installation of Ranger PolicyManager Web Application is completed.

表示已完成安装。

2.4 启动ranger-admin

#  进入ews目录
cd ews 
# 启动服务
./ranger-admin-services.sh  start
/**  正常情况下出现如下结果*/ 
Starting Apache Ranger Admin Service

Apache Ranger Admin Service with pid 236275 has started.

#  查看是否启动
ps -ef|grep ranger
或
netstat -lntp|grep 6080

2.5 web端验证

启动完成后,可以用web端登录验证,默认端口为6080,默认用户名密码均为admin

登录后,结果如下

3. 配置hdfs插件

3.1 在ranger-admim上配置一个服务

注意配置的服务名与底层配置文件里配置的服务名一致

3.2 修改配置文件

# 解压
tar -zxvf ranger-1.2.0-hdfs-plugin.tar.gz

# 进入目录
cd  /opt/apache-ranger-1.2.0/target/ranger-1.2.0-hdfs-plugin 

# 修改install.properties

vim install.properties

/**  主要修改如下内容*/
# Example:
# POLICY_MGR_URL=http://policymanager.xasecure.net:6080
#

POLICY_MGR_URL=http://192.168.1.17:6080

#
# This is the repository name created within policy manager
#
# Example:
# REPOSITORY_NAME=hadoopdev
#
REPOSITORY_NAME=hadoopdev

#
# Set hadoop home when hadoop program and Ranger HDFS Plugin are not in the
# same path.
#
COMPONENT_INSTALL_DIR_NAME=/opt/software/hadoop/hadoop-3.2.0

3.3 启动插件

./enable-hdfs-plugin.sh 
/** 结果如下*/
Custom user and group is available, using custom user and group.
+ Tue Aug 20 18:04:14 CST 2019 : hadoop: lib folder=/opt/software/hadoop/hadoop-3.2.0/share/hadoop/hdfs/lib conf folder=/opt/software/hadoop/hadoop-3.2.0/etc/hadoop
+ Tue Aug 20 18:04:14 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/hdfs-site.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.hdfs-site.xml.20190820-180414 ...
+ Tue Aug 20 18:04:14 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-hdfs-audit.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-hdfs-audit.xml.20190820-180414 ...
+ Tue Aug 20 18:04:15 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-hdfs-security.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-hdfs-security.xml.20190820-180414 ...
+ Tue Aug 20 18:04:15 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-policymgr-ssl.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-policymgr-ssl.xml.20190820-180414 ...
+ Tue Aug 20 18:04:16 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20190820180416 ...
Ranger Plugin for hadoop has been enabled. Please restart hadoop to ensure that changes are effective.

网页上测试成功后即可

其他组件的测试和hdfs类似,可自行尝试。如果大家使用过程中编译或配置有问题,可以添加微信或公众号和我沟通讨论。

本文分享自微信公众号 - 数据库干货铺(database_gjc),作者:懂点IT的耿小厨

原文出处及转载信息见文内详细说明,如有侵权,请联系 yunjia_community@tencent.com 删除。

原始发表时间:2019-08-20

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • ORDER BY导致未按预期使用索引

    在MySQL中经常出现未按照理想情况使用索引的情况,今天记录一种Order by语句的使用导致未按预期使用索引的情况。

    July
  • MySQL敏感数据加密及解密

    大数据时代的到来,数据成为企业最重要的资产之一,数据加密的也是保护数据资产的重要手段。本文主要在结合学习通过MySQL函数及Python加密方法来演示数据加密的...

    July
  • 数据库优化这些方法你都知道么

    上周针对MySQL数据库如何发现慢SQL、如何优化及预防进行了一次分享,其中主要的理论内容先分享给大家,案例因涉及业务信息,待修改后于后期逐步分享。

    July
  • Android 5秒学会使用手势解锁功能

    本文讲述的是一个手势解锁的库,可以定制显示隐藏宫格点、路径、并且带有小九宫格显示图,和震动!让你学会使用这个简单,高效的库!

    砸漏
  • 强化学习方法汇总,以及他们的区别

    了解强化学习中常用到的几种方法,以及他们的区别, 对我们根据特定问题选择方法时很有帮助. 强化学习是一个大家族, 发展历史也不短, 具有很多种不同方法. 比如...

    机器人网
  • Android实现双击TitleBar回顶部的功能示例代码

    本文介绍的内容是偶然发现的这个功能,就给移过来了,整理了一下,也是一个类就实现的,使用很方便

    砸漏
  • Java工具类之浮点精确计算

    二十三年蝉
  • 原来有这么多清北毕业生去了华为!

    意外的第一点,是清北毕业生出国人数并不像网上传言那么高,大部分学子选择留在祖国继续发展。

    肉眼品世界
  • 马哥金牌分享 | Docker在云家政的应用

    本文是由运维专家、马哥教育特约讲师Fighter分享的《Docker在云家政的应用》的文字整理版。 大家晚上好,我是Fighter,今晚由我给大家分享《Dock...

    小小科
  • [DeeplearningAI笔记]序列模型1.5-1.6不同类型的循环神经网络/语言模型与序列生成

    通过前向传播使用 Softmax 计算字典中各个单词出现的概率 输出字典中所有词的概率

    DrawSky

扫码关注云+社区

领取腾讯云代金券