lustre中服务通过MGS加载的Device和服务的内核模块
lustre中服务通过MGS加载的Device和服务的内核模块
用户4700054
发布于 2022-08-17 13:10:13
发布于 2022-08-17 13:10:13
lustre系统后端磁盘初始化脚本
// mdt和mgt共享一个磁盘，因为mgt仅仅需要100M空间，单独一个磁盘会浪费空间
[root@CentOS-Lustre-Server ~]$ mkfs.lustre --reformat --fsname=perrynfs --mgs --mdt --index=0 /dev/sdb
// 初始化ost，这里在ost对应的磁盘写入了mgs的地址
[root@CentOS-Lustre-Server ~]$ mkfs.lustre --ost --reformat --fsname perrynfs --index 0 --mgsnode CentOS-Lustre-Server@tcp0 /dev/sdc
// 客户端挂载时候制定了mgs的地址
[root@CentOS-Lustre-Client ~]$ mount -t lustre CentOS-Lustre-Server@tcp0:/perrynfs /mnt/lustre/
lustre服务中每个服务加载的Device
- lustre不管是客户端还是
MDS、MGS、OSS服务 都是通过加载device来组件整个服务，那么针对MDS/OSS/MGS在每个MGS节点上输出每个服务的device的信息
//挂载mgt和mdt节点
[root@CentOS-Lustre-Server ~]$ mount -t ldiskfs /dev/sdb /mnt/mgt_mdt
// 查看当前MGS端文件系统的配置信心
[root@CentOS-Lustre-Server ~]$ ls /mnt/mgt_mdt/CONFIGS/
mountdata nodemap osd-ldiskfs-client params perrynfs-client perrynfs-MDT0000 perrynfs-OST0000 perrynfs-OST0001 perrynfs-OST0002
- 查看
客户端通过挂载参数解析,初始化MDC连接到MGS中，MGS通过lustre log推送给客户端，客户端根据lustre log来加载自己的device，这些device如下
[root@CentOS-Lustre-Server ~]$ lctl --device MGS llog_print perrynfs-client
- { index: 2, event: attach, device: perrynfs-clilov, type: lov, UUID: perrynfs-clilov_UUID }
- { index: 3, event: setup, device: perrynfs-clilov, UUID: }
- { index: 6, event: attach, device: perrynfs-clilmv, type: lmv, UUID: perrynfs-clilmv_UUID }
- { index: 7, event: setup, device: perrynfs-clilmv, UUID: }
- { index: 10, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 11, event: attach, device: perrynfs-MDT0000-mdc, type: mdc, UUID: perrynfs-clilmv_UUID }
- { index: 12, event: setup, device: perrynfs-MDT0000-mdc, UUID: perrynfs-MDT0000_UUID, node: 10.211.55.6@tcp }
- { index: 13, event: add_mdc, device: perrynfs-clilmv, mdt: perrynfs-MDT0000_UUID, index: 0, gen: 1, UUID: perrynfs-MDT0000-mdc_UUID }
- { index: 16, event: new_profile, name: perrynfs-client, lov: perrynfs-clilov, lmv: perrynfs-clilmv }
- { index: 19, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 20, event: attach, device: perrynfs-OST0000-osc, type: osc, UUID: perrynfs-clilov_UUID }
- { index: 21, event: setup, device: perrynfs-OST0000-osc, UUID: perrynfs-OST0000_UUID, node: 10.211.55.6@tcp }
- { index: 22, event: add_osc, device: perrynfs-clilov, ost: perrynfs-OST0000_UUID, index: 0, gen: 1 }
- { index: 25, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 26, event: attach, device: perrynfs-OST0001-osc, type: osc, UUID: perrynfs-clilov_UUID }
- { index: 27, event: setup, device: perrynfs-OST0001-osc, UUID: perrynfs-OST0001_UUID, node: 10.211.55.6@tcp }
- { index: 28, event: add_osc, device: perrynfs-clilov, ost: perrynfs-OST0001_UUID, index: 1, gen: 1 }
- { index: 31, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 32, event: attach, device: perrynfs-OST0002-osc, type: osc, UUID: perrynfs-clilov_UUID }
- { index: 33, event: setup, device: perrynfs-OST0002-osc, UUID: perrynfs-OST0002_UUID, node: 10.211.55.6@tcp }
- { index: 34, event: add_osc, device: perrynfs-clilov, ost: perrynfs-OST0002_UUID, index: 2, gen: 1 }OSS端是初始化磁盘把MGS的地址写入磁盘，挂载磁盘时候，参数初始化MGC连接到MGS,通过lustre log来告诉OSS需要加载那些内核模块,从如下信息得知OSS加载了一个deviceobdfilter.
[root@CentOS-Lustre-Server ~]$ lctl --device MGS llog_print perrynfs-OST0000
- { index: 2, event: attach, device: perrynfs-OST0000, type: obdfilter, UUID: perrynfs-OST0000_UUID }
- { index: 3, event: setup, device: perrynfs-OST0000, UUID: dev, node: type, options: f }
[root@CentOS-Lustre-Server ~]$MDS端是初始化磁盘把MGS的地址写入磁盘,解析挂载参数初始化MGC连接到MGS,通过lustre log来告诉MDS需要加载那些device
[root@CentOS-Lustre-Server ~]$ lctl --device MGS llog_print perrynfs-MDT0000
- { index: 2, event: attach, device: perrynfs-MDT0000-mdtlov, type: lov, UUID: perrynfs-MDT0000-mdtlov_UUID }
- { index: 3, event: setup, device: perrynfs-MDT0000-mdtlov, UUID: }
- { index: 6, event: attach, device: perrynfs-MDT0000, type: mdt, UUID: perrynfs-MDT0000_UUID }
- { index: 7, event: new_profile, name: perrynfs-MDT0000, lov: perrynfs-MDT0000-mdtlov }
- { index: 8, event: setup, device: perrynfs-MDT0000, UUID: perrynfs-MDT0000_UUID, node: 0, options: perrynfs-MDT0000-mdtlov, failout: f }
- { index: 11, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 12, event: attach, device: perrynfs-OST0000-osc-MDT0000, type: osc, UUID: perrynfs-MDT0000-mdtlov_UUID }
- { index: 13, event: setup, device: perrynfs-OST0000-osc-MDT0000, UUID: perrynfs-OST0000_UUID, node: 10.211.55.6@tcp }
- { index: 14, event: add_osc, device: perrynfs-MDT0000-mdtlov, ost: perrynfs-OST0000_UUID, index: 0, gen: 1 }
- { index: 17, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 18, event: attach, device: perrynfs-OST0001-osc-MDT0000, type: osc, UUID: perrynfs-MDT0000-mdtlov_UUID }
- { index: 19, event: setup, device: perrynfs-OST0001-osc-MDT0000, UUID: perrynfs-OST0001_UUID, node: 10.211.55.6@tcp }
- { index: 20, event: add_osc, device: perrynfs-MDT0000-mdtlov, ost: perrynfs-OST0001_UUID, index: 1, gen: 1 }
- { index: 23, event: add_uuid, nid: 10.211.55.6@tcp(0x200000ad33706), node: 10.211.55.6@tcp }
- { index: 24, event: attach, device: perrynfs-OST0002-osc-MDT0000, type: osc, UUID: perrynfs-MDT0000-mdtlov_UUID }
- { index: 25, event: setup, device: perrynfs-OST0002-osc-MDT0000, UUID: perrynfs-OST0002_UUID, node: 10.211.55.6@tcp }
- { index: 26, event: add_osc, device: perrynfs-MDT0000-mdtlov, ost: perrynfs-OST0002_UUID, index: 2, gen: 1 }lustre每个服务加载的内核模块
本文参与 腾讯云自媒体分享计划，分享自微信公众号。
原始发表：2022-04-19，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读
目录