使用ceph-ansible部署Ceph Octopus
使用ceph-ansible部署Ceph Octopus
之前用的是 ceph-deploy 部署 ceph 集群，在官网的最新介绍中有如下描述：
ceph-deploy is no longer actively maintained. It is not tested on versions of Ceph newer than Nautilus. It does not support RHEL8, CentOS 8, or newer operating systems.
ceph-deploy 已经不在维护，并且在 ceph Nautilus 之后都没有很好的测试，不支持 RHEL8、CentOS8等系统。对比了 ceph-ansible 和 cephadm 这两个工具，最终选择的 ceph-ansible 作为部署工具。
主机角色
Hostname | Role | Admin/Public Network | Cluster Network | OS |
|---|---|---|---|---|
node01.ceph.local | ceph-ansible,mon,mgr,osd,rgw,mds,grafana | 192.168.198.131/24 | 172.20.1.131/24 | CentOS Linux release 8.3.2011 |
node02.ceph.local | mon,mgr,osd,rgw,mds | 192.168.198.132/24 | 172.20.1.132/24 | CentOS Linux release 8.3.2011 |
node03.ceph.local | mon,mgr,osd,rgw,mds | 192.168.198.133/24 | 172.20.1.133/24 | CentOS Linux release 8.3.2011 |
前期准备
包括配置IP、配置主机名、配置HOSTS文件、配置加速源（EPEL）、配置时钟同步、关闭 SELinux、关闭Firewalld、设置免密登录等，不再赘述。
Ansible配置
1.ansible安装
使用以下命令安装 ansible, ceph-ansible stable 5.x 需要 ansible 2.9 源中的版本满足需求，可以直接 yum 安装
# yum install ansible -y2.主机配置
编辑 /etc/ansible/hosts 加入以下内容
# vim /etc/ansible/hosts
[mons]
node01.ceph.local
node02.ceph.local
node03.ceph.local
[mgrs]
node01.ceph.local
node02.ceph.local
node03.ceph.local
[osds]
node01.ceph.local
node02.ceph.local
node03.ceph.local
[rgws]
node01.ceph.local
node02.ceph.local
node03.ceph.local
[mdss]
node01.ceph.local
node02.ceph.local
node03.ceph.local
[grafana-server]
node01.ceph.localceph-ansible配置
1.下载 ceph-ansible
下载 5.x 版本的 ceph-ansible
# wget https://github.com/ceph/ceph-ansible/archive/v5.0.3.tar.gz2.安装 ceph-ansible 依赖
# tar zxvf v5.0.3.tar.gz
# yum install python-pip
# pip install -r ceph-ansible/requirements.txt3.修改安装 ceph 配置
# cd ceph-ansible/group_vars/
# mv all.yml.sample all.yml
# grep -Ev '^#|^$' all.yml
---
dummy:
ceph_repository_type: repository
ceph_origin: repository
ceph_repository: community
ceph_mirror: https://mirrors.tuna.tsinghua.edu.cn/ceph/
ceph_stable_key: https://mirrors.tuna.tsinghua.edu.cn/ceph/keys/release.asc
ceph_stable_release: octopus
ceph_stable_repo: https://mirrors.tuna.tsinghua.edu.cn/ceph/rpm-15.2.7/el7/
monitor_interface: eth0
public_network: 192.168.198.0/24
cluster_network: 172.20.1.0/24
osd_objectstore: bluestore
radosgw_civetweb_port: 8080
radosgw_interface: eth0
dashboard_enabled: True
dashboard_admin_user: admin
dashboard_admin_password: [email protected]
grafana_admin_user: admin
grafana_admin_password: [email protected]4.osd配置
按照服务器配置的磁盘修改以下配置文件
# cp osds.yml.sample osds.yml
# grep -Ev '^#|^$' osds.yml
---
dummy:
devices:
- /dev/sdb
- /dev/sdc5.其他配置
复制其他的配置文件
# cp clients.yml.sample clients.yml
# cp mons.yml.sample mons.yml
# cp mgrs.yml.sample mgrs.yml
# cp rgws.yml.sample rgws.yml
# cd ../
# cp site.yml.sample site.yml开始安装
执行以下命令开始安装
# ansible-playbook site.yml完成安装后查看集群状态：
# ceph -s
cluster:
id: 6e344dd2-341a-4bb6-aafa-4299a0ebbe51
health: HEALTH_OK
services:
mon: 3 daemons, quorum node01.ceph.local,node02.ceph.local,node03.ceph.local (age 6m)
mgr: node01.ceph.local(active, since 40s), standbys: node02.ceph.local, node02.ceph.local
mds: cephfs:1 {0=node01.ceph.local=up:active} 2 up:standby
osd: 12 osds: 12 up (since 2m), 12 in (since 23m)
rgw: 3 daemons active (node01.ceph.local.rgw0, node02.ceph.local.rgw0, 3.ceph.local.rgw0)
data:
pools: 6 pools, 144 pgs
objects: 212 objects, 6.4 KiB
usage: 12 GiB used, 9.8 TiB / 9.8 TiB avail
pgs: 144 active+clean清理安装
如果安装出错，或者遇到其他问题，可以清理集群后再次尝试安装
# ansible-playbook infrastructure-playbooks/purge-cluster.yml