CentOS 5.9下安装配置Keepalived
Keepalived是由C语言编写的路由软件，是lvs的扩展项目，主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现。keepalived是一个类似于layer3, 4 & 7交换机制的软件，也就是我们平时说的第3层、第4层和第7层交换。本文主要介绍了其安装过程供大家参考。
1、获取安装软件
下载地址：http://www.keepalived.org/download.html
可以直接下载最新的版本1.2.13，也可以通过wget自动下载
# wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz
2、安装前提
a、确保ipvs已经被安装 #如下，执行ipvsadm检查ipvs是否已经安装(当前大多数Linux缺省已安装) [root@HKBO ~]# ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn
#检查当前加载的内核模块，看是否存在ip_vs模块 [root@HKBO ~]# lsmod|grep ip_vs ip_vs 122113 0
b、安装相应的依赖包 # yum -y install gcc gcc-c++ gcc-g77 ncurses-devel bison libaio-devel \ > cmake libnl* libpopt* popt-static openssl-devel
3、安装keepalived
[root@HKBO ~]# more /etc/issue CentOS release 5.9 (Final)
# mkdir /usr/local/keepalived # cd /usr/local/src # wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz # tar -xvf keepalived-1.2.13.tar.gz # cd keepalived-1.2.13 # ./configure --disable-fwmark --prefix=/usr/local/keepalived ............... Keepalived configuration ------------------------ Keepalived version : 1.2.13 Compiler : gcc Compiler flags : -g -O2 -DETHERTYPE_IPV6=0x86dd Extra Lib : -lssl -lcrypto -lcrypt Use IPVS Framework : Yes IPVS sync daemon support : Yes IPVS use libnl : No fwmark socket support : No Use VRRP Framework : Yes Use VRRP VMAC : No SNMP support : No SHA1 support : No Use Debug flags : No
# make && make install # ls /usr/local/keepalived/ #查看安装完成后，生成了4个目录 bin etc sbin share
# cd /usr/local/keepalived/etc [root@HKBO etc]# tree . |-- keepalived | |-- keepalived.conf #keepalived的配置样例 | `-- samples | |-- client.pem | |-- dh1024.pem | |-- keepalived.conf.HTTP_GET.port | |-- keepalived.conf.IPv6 | |-- keepalived.conf.SMTP_CHECK | |-- keepalived.conf.SSL_GET | |-- keepalived.conf.fwmark | |-- keepalived.conf.inhibit | |-- keepalived.conf.misc_check | |-- keepalived.conf.misc_check_arg | |-- keepalived.conf.quorum | |-- keepalived.conf.sample | |-- keepalived.conf.status_code | |-- keepalived.conf.track_interface | |-- keepalived.conf.virtual_server_group | |-- keepalived.conf.virtualhost | |-- keepalived.conf.vrrp | |-- keepalived.conf.vrrp.localcheck | |-- keepalived.conf.vrrp.lvs_syncd | |-- keepalived.conf.vrrp.routes | |-- keepalived.conf.vrrp.scripts | |-- keepalived.conf.vrrp.static_ipaddress | |-- keepalived.conf.vrrp.sync | |-- root.pem | `-- sample.misccheck.smbcheck.sh |-- rc.d | `-- init.d | `-- keepalived `-- sysconfig `-- keepalived
# cd /usr/local/keepalived/sbin # ./keepalived --help #获取keepalived的一些帮助 Usage: ./keepalived [OPTION...] -f, --use-file=FILE Use the specified configuration file -P, --vrrp Only run with VRRP subsystem -C, --check Only run with Health-checker subsystem -l, --log-console Log messages to local console -D, --log-detail Detailed log messages -S, --log-facility=[0-7] Set syslog facility to LOG_LOCAL[0-7] -V, --dont-release-vrrp Don't remove VRRP VIPs and VROUTEs on daemon stop -I, --dont-release-ipvs Don't remove IPVS topology on daemon stop -R, --dont-respawn Don't respawn child processes -n, --dont-fork Don't fork the daemon process -d, --dump-conf Dump the configuration data -p, --pid=FILE Use specified pidfile for parent process -r, --vrrp_pid=FILE Use specified pidfile for VRRP child process -c, --checkers_pid=FILE Use specified pidfile for checkers child process -v, --version Display the version number -h, --help Display this help message
# mkdir /etc/keepalived/ # cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/ # cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ # cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ # ln -s /usr/local/keepalived/sbin/keepalived /sbin/
# service keepalived restart Stopping keepalived: [FAILED] Starting keepalived: [ OK ]
# service keepalived status keepalived (pid 12092) is running...
# chkconfig keepalived on
# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:50:56:84:04:2c brd ff:ff:ff:ff:ff:ff inet 192.168.1.66/24 brd 192.168.1.255 scope global eth0 inet 192.168.200.16/32 scope global eth0 inet 192.168.200.17/32 scope global eth0 inet 192.168.200.18/32 scope global eth0 inet6 fe80::250:56ff:fe84:42c/64 scope link valid_lft forever preferred_lft forever 3: sit0: <NOARP> mtu 1480 qdisc noop link/sit 0.0.0.0 brd 0.0.0.0
4、有关keepalived.conf的配置项
[root@HKBO keepalived]# more keepalived.conf ! Configuration File for keepalived
#全局定义块，包含邮件定义部分， global_defs { notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from Alexandre.Cassen@firewall.loc smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id LVS_DEVEL //负载均衡器标识，在一个网络内，它应该是唯一的 }
#VRRP实例定义块，负责负载均衡器之间的失败切换 vrrp_instance VI_1 { state MASTER //只有MASTER和BACKUP 2种状态，主为MASTER，从为BACKUP，使用大写 interface eth0 //监控的网络接口 virtual_router_id 51 //同一实例下virtual_router_id必须相同 priority 100 //定义优先级，数字越大，优先级越高 advert_int 1 //MASTER与BACKUP负载均衡器之间同步检查的时间间隔，单位是秒 authentication { //验证类型和密码 auth_type PASS auth_pass 1111 } virtual_ipaddress { //虚拟ip地址virtual_ipaddress,可以定义多个 192.168.200.16 192.168.200.17 192.168.200.18 } }
#虚拟服务器定义块 virtual_server 192.168.200.100 443 { //定义虚拟服务器 delay_loop 6 //delay_loop，健康检查时间间隔，单位是秒 lb_algo rr //负载调度算法，这里设置为rr，即轮询算法，互联网应用常使用wlc或rr lb_kind NAT //负载均衡转发规则。一般包括DR,NAT,TUN3种，在我的方案中，都使用DR的方式 nat_mask 255.255.255.0 //子网掩码 persistence_timeout 50 //会话保持时间，单位是秒（可以适当延长时间以保持session） protocol TCP //转发协议类型，有tcp和udp两种 real_server 192.168.201.100 443 { //真实服务器IP及端口 weight 1 //默认为1,0为失效 SSL_GET { url { path / digest ff20ad2481f97b1754ef3e12ecd3a9cc } url { path /mrtg/ digest 9b3a0c85a887a256d6939da88aabd8cd } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }
virtual_server 10.10.10.2 1358 { delay_loop 6 lb_algo rr lb_kind NAT persistence_timeout 50 protocol TCP
sorry_server 192.168.200.200 1358
real_server 192.168.200.2 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl3/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } }
real_server 192.168.200.3 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334c } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334c } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }
5、配置主备keepalived
a、配置主备keepalived #主，以下主要列出keepalived.conf差异部分 vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.1.220 192.168.1.230 } }
#备，以下主要列出keepalived.conf差异部分 vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.1.220 192.168.1.230 } }
b、测试主备keepalived #启动主服务器上的keepalived服务 [root@MASTER ~]# service keepalived start Starting keepalived: [ OK ]
[root@SZ-SYS-APP01 ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:50:56:84:1f:37 brd ff:ff:ff:ff:ff:ff inet 192.168.1.65/24 brd 192.168.1.255 scope global eth0 inet 192.168.1.220/32 scope global eth0 #可以看到220和230的虚IP inet 192.168.1.230/32 scope global eth0 inet6 fe80::250:56ff:fe84:1f37/64 scope link valid_lft forever preferred_lft forever 3: sit0: <NOARP> mtu 1480 qdisc noop link/sit 0.0.0.0 brd 0.0.0.0
#启动备用服务器上的keepalived服务 [root@BACKUP ~]# service keepalived start Starting keepalived: [ OK ] #下面的查询结果中，虚IP220和230并为出现在备用服务器 [root@HKBO ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:50:56:84:04:2c brd ff:ff:ff:ff:ff:ff inet 192.168.1.66/24 brd 192.168.1.255 scope global eth0 inet6 fe80::250:56ff:fe84:42c/64 scope link valid_lft forever preferred_lft forever 3: sit0: <NOARP> mtu 1480 qdisc noop link/sit 0.0.0.0 brd 0.0.0.0
#关闭主服务上的keepalived服务，观察vip是否会漂移到备用服务 [root@MASTER ~]# service keepalived stop Stopping keepalived: [ OK ] [root@MASTER ~]# ip addr #关闭后，该命令可以看到220和230的虚IP已经不存在于主服务器 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:50:56:84:1f:37 brd ff:ff:ff:ff:ff:ff inet 192.168.1.65/24 brd 192.168.1.255 scope global eth0 inet6 fe80::250:56ff:fe84:1f37/64 scope link valid_lft forever preferred_lft forever 3: sit0: <NOARP> mtu 1480 qdisc noop link/sit 0.0.0.0 brd 0.0.0.0
#此时在备用服务器上查看到了漂移的vip地址 [root@BACKUP ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:50:56:84:04:2c brd ff:ff:ff:ff:ff:ff inet 192.168.1.66/24 brd 192.168.1.255 scope global eth0 inet 192.168.1.220/32 scope global eth0 inet 192.168.1.230/32 scope global eth0 inet6 fe80::250:56ff:fe84:42c/64 scope link valid_lft forever preferred_lft forever 3: sit0: <NOARP> mtu 1480 qdisc noop link/sit 0.0.0.0 brd 0.0.0.0
#如果再次启动主服务器上的keepalived服务，则会vip自动漂移到主服务器，不再验证。 #Author : Leshami #Blog ：http://www.linuxidc.com
6、配置双主keepalived
双主的keepalived方式实际上是互为主备，以避免单点故障 #配置主备keepalived MasterA: 192.168.1.65, VirtualIP: 192.168.1.220 MasterB: 192.168.1.66, VirtualIP: 192.168.1.230 #主，以下主要列出keepalived.conf差异部分 vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.1.220 } }
vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 52 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.1.230 } }
#备，以下主要列出keepalived.conf差异部分 vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.1.220 } }
vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 52 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.1.230 } }
#从上面的配置文件中可以看出，实际上是增加了一个vrrp实例 #验证过程略
7、编译时的错误No SO_MARK
#如果在编译时碰到了No SO_MARK错误，如下 # ./configure checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o ........... checking for nl_socket_alloc in -lnl-3... no checking for nl_socket_modify_cb in -lnl... no configure: WARNING: keepalived will be built without libnl support. checking for kernel version... 2.6.18 checking for IPVS syncd support... yes checking for kernel macvlan support... no checking whether SO_MARK is declared... no configure: error: No SO_MARK declaration in headers
#可以通过增加disable-fwmark参数来解决 --disable-fwmark compile without SO_MARK support