Alluxio users, when accessing data on COS, HDFS, CHDFS via the existing unified namespace, or when utilizing transparent URLs to access cached data within Alluxio, may encounter situations devoid of authentication. In other words, any user possessing the corresponding URI can access the data. To address such scenarios, Alluxio in the cloud has enhanced the authentication process in conjunction with Ranger and CosRanger.
Note
To accommodate the authentication feature, please ensure the integration of the following components within the cluster:
Should Alluxio only have HDFS mounted, the integration of the Ranger component is required.
Should Alluxio have COS and CHDFS mounted, the integration of the CosRanger component is necessitated.
Supported Versions
Supported Service Component Version: Alluxio 2.8.0.
Product Version: Standard Hadoop 3.x, EMR-V3.4.0.
Configure authentication
Preliminary Configuration
#Add new configuration item for Hive component ranger-hive-security.xmlranger.plugin.hive.urlauth.filesystem.schemes==hdfs:,file:,wasb:,adl:,alluxio:#Add new configuration item for Presto component hive.propertieshive.hdfs.authentication.type=NONEhive.metastore.authentication.type=NONEhive.hdfs.impersonation.enabled=truehive.metastore.thrift.impersonation.enabled=true
Note
The aforementioned preliminary configurations should be set according to the existing components of the cluster.
HDFS Authentication
Create symbolic links for Ranger-related configuration files:
[hadoop@172 conf]$ pwd/usr/local/service/alluxio/conf[hadoop@172 conf]$ ln -s /usr/local/service/hadoop/etc/hadoop/ranger-hdfs-audit.xmlranger-hdfs-audit.xml[hadoop@172 conf]$ ln -s /usr/local/service/hadoop/etc/hadoop/ranger-hdfs-security.xml ranger-hdfs-security.xml
Configuration of alluxio-site.properties
It is recommended to use the EMR console for cluster-level configuration distribution.
# Authentication switch (default is false)alluxio.security.authorization.plugins.enabled=truealluxio.security.authorization.plugin.name=rangeralluxio.security.authorization.plugin.paths=/usr/local/service/alluxio/confalluxio.underfs.security.authorization.plugin.name=rangeralluxio.underfs.security.authorization.plugin.paths=/usr/local/service/alluxio/confalluxio.master.security.impersonation.hadoop.users=*alluxio.security.login.impersonation.username=_HDFS_USER_
Note
Upon completion of the distribution, it is necessary to restart the Alluxio service.
COS and CHDFS Authentication
#Add new configuration items to core-site.xmlfs.ofs.ranger.enable.flag=true
Configuration of alluxio-site.properties
It is recommended to use the EMR console for cluster-level configuration distribution.
# Authentication switch (default is false)# Authentication switch (default is false)alluxio.security.authorization.plugins.enabled=truealluxio.security.authorization.plugin.name=rangeralluxio.security.authorization.plugin.paths=/usr/local/service/alluxio/confalluxio.underfs.security.authorization.plugin.name=rangeralluxio.underfs.security.authorization.plugin.paths=/usr/local/service/alluxio/confalluxio.cos.qcloud.object.storage.ranger.service.config.dir=/usr/local/service/cosranger/confalluxio.master.security.impersonation.hadoop.users=*alluxio.security.login.impersonation.username=_HDFS_USER_# The default number of retry attempts is five.alluxio.cos.qcloud.object.storage.permission.check.max.retry=5
Note
Upon completion of the distribution, it is necessary to restart the Alluxio service.