Suse 11下多路径及udev配置
最近给客户基于SuSe 11 SP3下多路径部署Oracle 10g RAC。SuSe 11下用10g，也算一朵奇葩，连篇文档都比较难找，谁叫Oracle太贵呢。下面主要是描述了在该环境下如何去配置多路径。由于10g下的ocr与votingdisk不能直接存放到asm磁盘，所以依旧要使用raw设备方式来保存。下文供大家参考。
一、查看当前scsi设备及获取设备wwid #使用 cat /proc/partitions或者/sbin/sfdisk命令查看当前系统中的scsi设备 suse11a:~ # more /proc/partitions major minor #blocks name
8 0 877264896 sda 8 1 200812 sda1 8 2 104856255 sda2 8 3 104856255 sda3 8 4 667340100 sda4 8 16 2097152000 sdb 8 17 2097150976 sdb1 ..... 8 240 1048576 sdp 65 0 1048576 sdq ..... 65 48 2097152000 sdt 65 49 2097150976 sdt1 253 0 2097152000 dm-0 253 1 2097152000 dm-1 253 2 1751121920 dm-2 253 3 1048576 dm-3 253 4 1048576 dm-4 253 5 1751120896 dm-5 253 6 2097150976 dm-6 253 7 2097150976 dm-7 #获取多路径设备的wwid，使用下面的命令来获取当前系统中多路径设备的wwid for i in `cat /proc/partitions | awk {'print $4'} |grep sd`; do echo "### $i: `/lib/udev/scsi_id --whitelist --replace-whitespace /dev/$i`"; done
### sda: 3600605b007030d301a192f2d13c017a0 ### sda1: 3600605b007030d301a192f2d13c017a0 ..................... ### sdt: 360022a11000cb6b10a902db200000000 ### sdt1: 360022a11000cb6b10a902db200000000 #获取后，过滤本地磁盘及重复的wwid
二、配置mulitpatch.conf文件 #配置mulitpatch.conf文件的主要目的是用于使用有意义，用户自定义的设备名称，持久化多路径设备 #这个步骤是与配置单路径设备最大的区别 #下面是已经配置过后的multipath.conf文件 suse11a:~ # cp /etc/multipath.conf /etc/multipath.conf.bak suse11a:~ # vi /etc/multipath.conf suse11a:~ # grep -v ^# /etc/multipath.conf defaults { user_friendly_names yes } blacklist { devnode ^hd[a-z] } multipaths { multipath { wwid 36200bc710099b9190ba6fc350000000a alias mpatha }
multipath { wwid 36200bc710099b9190ba7109b0000000b alias mpathb } multipath { wwid 36200bc710099b9190ab1155300000009 alias mpathc } multipath { wwid 360022a11000cb6b10a902db200000000 alias mpathd } multipath { wwid 360022a11000cb6b10a9116d000000002 alias mpathe } }
三、校验多路径设备 #使用下面的命令用于多路径配置生效 suse11a:~ # service multipathd stop suse11a:~ # service multipathd start
#下面校验多路径设备 suse11a:~ # multipath -ll | grep mpath mpathe (360022a11000cb6b10a9116d000000002) dm-1 HUAWEI ,S5500T mpathd (360022a11000cb6b10a902db200000000) dm-0 HUAWEI ,S5500T mpathc (36200bc710099b9190ab1155300000009) dm-2 HUAWEI ,S2600T mpathb (36200bc710099b9190ba7109b0000000b) dm-4 HUAWEI ,S2600T mpatha (36200bc710099b9190ba6fc350000000a) dm-3 HUAWEI ,S2600T
#下面是设备的详细信息 suse11a:~ # multipath -ll mpathe (360022a11000cb6b10a9116d000000002) dm-1 HUAWEI ,S5500T size=2.0T features='0' hwhandler='0' wp=rw |-+- policy='service-time 0' prio=1 status=active | `- 5:0:0:1 sdc 8:32 active ready running |-+- policy='service-time 0' prio=1 status=enabled | `- 5:0:3:1 sdk 8:160 active ready running |-+- policy='service-time 0' prio=1 status=enabled | `- 6:0:2:1 sds 65:32 active ready running `-+- policy='service-time 0' prio=1 status=enabled `- 6:0:3:1 sdu 65:64 active ready running mpathd (360022a11000cb6b10a902db200000000) dm-0 HUAWEI ,S5500T size=2.0T features='0' hwhandler='0' wp=rw |-+- policy='service-time 0' prio=1 status=active | `- 5:0:0:0 sdb 8:16 active ready running |-+- policy='service-time 0' prio=1 status=enabled | `- 5:0:3:0 sdj 8:144 active ready running |-+- policy='service-time 0' prio=1 status=enabled | `- 6:0:2:0 sdr 65:16 active ready running `-+- policy='service-time 0' prio=1 status=enabled `- 6:0:3:0 sdt 65:48 active ready running .........
suse11a:~ # dmsetup ls |sort mpatha (253:3) mpathb (253:4) mpathc (253:2) mpathc_part1 (253:5) mpathd (253:0) mpathd_part1 (253:6) mpathe (253:1) mpathe_part1 (253:7) #以下命令均可以用于校验多路径设备 # ll /dev/disk/by-id/ # ls -l /dev/dm-* # ll /dev/mapper/
#/dev/dm-*下面的为动态名，即系统重启后会发生变化，此部分用于os 内部使用。 #/dev/mapper/下面查看到的内容为持久化名称，也即是我们在mulitpatch.conf文件中定义的。
四、配置RAW #在Oracle 10g R2 rac安装的过程中，OUI不能够校验共享的快涉笔，因此需要使用raw。Oracle 11g R1后可以直接使用多路径设备。 #使用udev来管理raw，编辑raw文件，如下 suse11a:~ # cat /etc/raw | grep -v \# raw1:mapper/mpatha raw2:mapper/mpathb raw3:mapper/mpathc raw4:mapper/mpathd raw5:mapper/mpathe
#重启raw以及配置开机启动raw suse11a:~ # rcraw start suse11a:~ # chkconfig raw on suse11a:~ # raw -qa /dev/raw/raw1: bound to major 253, minor 3 /dev/raw/raw2: bound to major 253, minor 4 /dev/raw/raw3: bound to major 253, minor 2 /dev/raw/raw4: bound to major 253, minor 0 /dev/raw/raw5: bound to major 253, minor 1
#测试raw的可访问性 # dd if=/dev/zero of=/dev/raw/raw1 bs=1024 count=100 100+0 records in 100+0 records out 102400 bytes (102 kB) copied, 0.762352 seconds, 134 kB/s
#配置权限，该权限用于oracle用户访问裸设备 # Author : Leshami # Blog : http://blog.csdn.net/leshami suse11a:~ # more /etc/udev/rules.d/99-raw.rules KERNEL=="raw[1-2]*", GROUP="oinstall", MODE="664" KERNEL=="raw[3-5]*", OWNER="oracle", GROUP="oinstall", MODE="664"
五、安装clusterware #安装clusterware时，关于ocr和votingdisk，选择/dev/raw/raw1，/dev/raw/raw2，如果有配置冗余，此处应该为多个裸设备。 #校验安转好clusterware后裸设备的使用情况 suse11a:oracle:ora10g1 > ocrcheck Status of Oracle Cluster Registry is as follows : Version : 2 Total space (kbytes) : 1048296 Used space (kbytes) : 4592 Available space (kbytes) : 1043704 ID : 892902908 Device/File Name : /dev/raw/raw1 Device/File integrity check succeeded
Device/File not configured
Cluster registry integrity check succeeded
suse11a:oracle:ora10g1 > crsctl query css votedisk 0. 0 /dev/raw/raw2
located 1 votedisk(s). #对于ASM磁盘可以通过dbca来创建，直接使用/dev/raw/raw*以及/dev/mapper/*。当然suse 11支持asmlib，也可以通过asmlib方式来访问asm磁盘。