我们在之前的讨论中介绍了Docker,那么接下来,我们将手把手带你搭建一个Alluxio集群。
首先要创建一个带有所需软件基本映像,并开放其SSH(22端口)。同时,创建Alluxio master所需的Hadoop用户,以便通过SSH启动进程。
下一步是创建安装Alluxio所需的Docker文件:
然后,创建具有集群(Master和Worker)信息的Docker-Compose文件。在这个的例子中,我们假设有一个master和两个worker。
version: '2'
services:
#############################
# Master1
#############################
master1:
container_name: master1
domainname: master1.dev
hostname: master1
image: alluxio_base
working_dir: /opt/apache/
restart: always
ports:
- "32000:22"
- "32001:19999"
volumes:
- ./master1:/apacheapps/data/alluxio/master1
# - ./master1/conf:/opt/apache/alluxio/conf
# - ./master1/logs:/opt/apache/alluxio/logs:rw
mem_limit: 1000000000
#############################
# worker1
#############################
worker1:
container_name: worker1
domainname: worker1.dev
hostname: worker1
image: alluxio_base
working_dir: /opt/apache/
restart: always
ports:
- "32002:22"
- "32003:30000"
- "32004:19999"
volumes:
- ./worker1:/apacheapps/data/alluxio/worker1
# - ./worker1/conf:/opt/apache/alluxio/conf
# - ./worker1/logs:/opt/apache/alluxio/logs:rw
- ./worker1/ramdisk:/mnt/ramdisk
mem_limit: 1000000000
#############################
# worker2
#############################
worker2:
container_name: worker2
domainname: worker2.dev
hostname: worker2
image: alluxio_base
working_dir: /opt/apache/
restart: always
ports:
- "32005:22"
- "32006:30000"
- "32007:19999"
volumes:
- ./worker2:/apacheapps/data/alluxio/worker2
# - ./worker2/conf:/opt/apache/alluxio/conf
# - ./worker2/logs:/opt/apache/alluxio/logs:rw
- ./worker1/ramdisk:/mnt/ramdisk
mem_limit: 1000000000
接下来,用 docker-compose up -d
命令启动容器。连接到主容器后,需要先sudo到Hadoop用户(sudo su - hadoop
),然后生成SSH密钥(ssh-keygen
),并将密钥(ssh-copy-id
)复制到workers(/ opt / apache / alluxio / workers
)
最后一步是创建Alluxio所需的配置文件,并启动集群。
cd /opt/apache/alluxio/bin。
./alluxio bootstrapConf master1
./alluxio copyDir ../conf
./alluxio format
./alluxio-start.sh all NoMount
下图就是最后群集的屏幕截图。