redis 3.0尝鲜

1.安装redis

wget  http://download.redis.io/releases/redis-3.0.5.tar.gz

tar zxf  redis-3.0.5.tar.gz  -C /export/server/

cd /export/servers/redis-3.0.5

make && make install 

mkdir conf;cp redis.conf conf/

2.redis集群模式配置

2.1redis主配置文件：

vim  /export/servers/redis-3.0.5/conf/redis.conf

daemonize yes pidfile /var/run/redis.pid tcp-backlog 511 timeout 0 tcp-keepalive 0 loglevel notice logfile "" databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb dir /export/data/redis slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 appendonly yes appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 cluster-enabled yes cluster-node-timeout 15000 cluster-migration-barrier 1 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-entries 512 list-max-ziplist-value 64 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes

2.2 redis实例节点配置

vim redis_7001.conf

#包含通用配置 include /export/servers/redis-3.0.5/conf/redis.conf #监听tcp端口 port 7001 #最大可用内存 maxmemory 100m #内存耗尽时采用的淘汰策略: # volatile-lru -> remove the key with an expire set using an LRU algorithm # allkeys-lru -> remove any key accordingly to the LRU algorithm # volatile-random -> remove a random key with an expire set # allkeys-random -> remove a random key, any key # volatile-ttl -> remove the key with the nearest expire time (minor TTL) # noeviction -> don't expire at all, just return an error on write operations maxmemory-policy allkeys-lru #aof存储文件 appendfilename "appendonly-7001.aof" #rdb文件,只用于动态添加slave过程 dbfilename dump-7001.rdb #cluster配置文件(启动自动生成) cluster-config-file nodes-7001.conf #部署在同一机器的redis实例，把<span style="font-size: 1em; line-height: 1.5;">auto-aof-rewrite搓开，防止瞬间fork所有redis进程做rewrite,占用大量内存</span> auto-aof-rewrite-percentage 80-100

创建多个redis实例配置

for i in `seq 2 6`;do cp redis_7001.conf redis_700$i.conf;done

for i in `seq 2 6`;do sed -i s/7001/700$i/g redis_700$i.conf;done

3.redis 实例启动(配置文件中定义了集群模式)

# cat redis_cluster.sh

#!/bin/bash for i in `seq 1 6`; do /export/servers/redis-3.0.5/src/redis-server /export/servers/redis-3.0.5/conf/redis_700$i.conf > /export/data/redis/redis-700$i.log 2>&1 & done

3.1 查看redis启动实例：

# ps -ef | grep redis

root 22748 1 0 10:59 ? 00:00:01 /export/servers/redis-3.0.5/src/redis-server *:7001 [cluster] root 22749 1 0 10:59 ? 00:00:01 /export/servers/redis-3.0.5/src/redis-server *:7006 [cluster] root 22750 1 0 10:59 ? 00:00:01 /export/servers/redis-3.0.5/src/redis-server *:7004 [cluster] root 22751 1 0 10:59 ? 00:00:01 /export/servers/redis-3.0.5/src/redis-server *:7003 [cluster] root 22752 1 0 10:59 ? 00:00:01 /export/servers/redis-3.0.5/src/redis-server *:7005 [cluster] root 22753 1 0 10:59 ? 00:00:01 /export/servers/redis-3.0.5/src/redis-server *:7002 [cluster] root 26628 8683 0 11:20 pts/0 00:00:00 grep redis

3.2 检查集群信息，因为还没有创建集群，没有设置主从关系呢。

 /export/servers/redis-3.0.5/src/redis-trib.rb check 127.0.0.1:7002

创建redis集群，自动设置主从关系

：需要注意的是，redis使用ruby写的，yum install ruby rubygems;gem install redis

# ./redis-trib.rb create --replicas 1 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006

>>> Creating cluster Connecting to node 127.0.0.1:7001: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7003: OK Connecting to node 127.0.0.1:7004: OK Connecting to node 127.0.0.1:7005: OK Connecting to node 127.0.0.1:7006: OK >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 Adding replica 127.0.0.1:7004 to 127.0.0.1:7001 Adding replica 127.0.0.1:7005 to 127.0.0.1:7002 Adding replica 127.0.0.1:7006 to 127.0.0.1:7003 M: 608a43079e6f61dfc8200c29fbad1c4cd721b06f 127.0.0.1:7001 slots:0-5460 (5461 slots) master M: 6935de6f15a45ca0aa0d6c785361e85bb3593585 127.0.0.1:7002 slots:5461-10922 (5462 slots) master M: 58d18867c2c17a9f9b30d9e5d62734878892179a 127.0.0.1:7003 slots:10923-16383 (5461 slots) master S: 82b8684f0b35f0147fed50254853898193fe0336 127.0.0.1:7004 replicates 608a43079e6f61dfc8200c29fbad1c4cd721b06f S: b230c9430a636f2421c323bb6cccda1c4dff16b7 127.0.0.1:7005 replicates 6935de6f15a45ca0aa0d6c785361e85bb3593585 S: 584e9035655d0de8dd4047c685363c2e5634e4e0 127.0.0.1:7006 replicates 58d18867c2c17a9f9b30d9e5d62734878892179a Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join..... >>> Performing Cluster Check (using node 127.0.0.1:7001) M: 608a43079e6f61dfc8200c29fbad1c4cd721b06f 127.0.0.1:7001 slots:0-5460 (5461 slots) master M: 6935de6f15a45ca0aa0d6c785361e85bb3593585 127.0.0.1:7002 slots:5461-10922 (5462 slots) master M: 58d18867c2c17a9f9b30d9e5d62734878892179a 127.0.0.1:7003 slots:10923-16383 (5461 slots) master M: 82b8684f0b35f0147fed50254853898193fe0336 127.0.0.1:7004 slots: (0 slots) master replicates 608a43079e6f61dfc8200c29fbad1c4cd721b06f M: b230c9430a636f2421c323bb6cccda1c4dff16b7 127.0.0.1:7005 slots: (0 slots) master replicates 6935de6f15a45ca0aa0d6c785361e85bb3593585 M: 584e9035655d0de8dd4047c685363c2e5634e4e0 127.0.0.1:7006 slots: (0 slots) master replicates 58d18867c2c17a9f9b30d9e5d62734878892179a [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.

默认redis集群中有16384个slot插槽，会随机的分配个三个主节点

• 给定 redis-trib.rb 程序的命令是 create ， 这表示我们希望创建一个新的集群。
• 选项 --replicas 1 表示我们希望为集群中的每个主节点创建一个从节点。
• 之后跟着的其他参数则是实例的地址列表， 我们希望程序使用这些地址所指示的实例来创建新集群。

4. 测试集群性能：

# ./redis-cli -c -p 7001

127.0.0.1:7001> set xxb Andy_xu

-> Redirected to slot [15549] located at 127.0.0.1:7003 OK 127.0.0.1:7003> exit

# ./redis-cli -c -p 7002

127.0.0.1:7002> get xxb

-> Redirected to slot [15549] located at 127.0.0.1:7003

“Andy_xu"

127.0.0.1:7003> get xxb

“Andy_xu"

127.0.0.1:7003> exit

# ./redis-cli -c -p 7005

127.0.0.1:7005> get xxb

-> Redirected to slot [15549] located at 127.0.0.1:7003

“Andy_xu"

127.0.0.1:7003>

现象解释：因为redis集群模式数据默认都是存放在16384slot里面的，算法会通过key计算，将数据存放在那个slot，而slot又是存放在某个redis实例的，所以我们在7001实例上查找key，会自动将数据定向到7003的一个插槽，因为xxb这个key最终是存放在7003实例的一个slot上的，因此每个节点去查找数据，最终会去插槽里面查找。

5. redis集群扩容

5.1 新加一个redis主节点：需要注意的是，现在7007节点只是添加到集群里，但是还是一个空的节点，没有分配插槽，所以需要对新的集群进行重新分片resharding

# ./redis-trib.rb add-node 127.0.0.1:7007 127.0.0.1:7001

>>> Adding node 127.0.0.1:7007 to cluster 127.0.0.1:7001 Connecting to node 127.0.0.1:7001: OK Connecting to node 127.0.0.1:7006: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7005: OK Connecting to node 127.0.0.1:7004: OK Connecting to node 127.0.0.1:7003: OK >>> Performing Cluster Check (using node 127.0.0.1:7001) M: 608a43079e6f61dfc8200c29fbad1c4cd721b06f 127.0.0.1:7001 slots:0-5460 (5461 slots) master 1 additional replica(s) S: 584e9035655d0de8dd4047c685363c2e5634e4e0 127.0.0.1:7006 slots: (0 slots) slave replicates 58d18867c2c17a9f9b30d9e5d62734878892179a M: 6935de6f15a45ca0aa0d6c785361e85bb3593585 127.0.0.1:7002 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: b230c9430a636f2421c323bb6cccda1c4dff16b7 127.0.0.1:7005 slots: (0 slots) slave replicates 6935de6f15a45ca0aa0d6c785361e85bb3593585 S: 82b8684f0b35f0147fed50254853898193fe0336 127.0.0.1:7004 slots: (0 slots) slave replicates 608a43079e6f61dfc8200c29fbad1c4cd721b06f M: 58d18867c2c17a9f9b30d9e5d62734878892179a 127.0.0.1:7003 slots:10923-16383 (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. Connecting to node 127.0.0.1:7007: OK >>> Send CLUSTER MEET to node 127.0.0.1:7007 to make it join the cluster. [OK] New node added correctly.

5.2 重新分片，将16384slot进行实例分配

# ./redis-trib.rb reshard 127.0.0.1:7007  给7007重新分片

[OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. How many slots do you want to move (from 1 to 16384)? 500         选择要移动的slot What is the receiving node ID? 608a43079e6f61dfc8200c29fbad1c4cd721b06f 选择接受slot的节点(应该是新加进去的7007的) Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.

Source node #1:all     选择重新分配master的slot

Ready to move 500 slots. Source nodes: M: 48e6cdd7355b49a63ab9f29fdd2bf7a7738505b5 127.0.0.1:7007 slots: (0 slots) master 0 additional replica(s) M: 58d18867c2c17a9f9b30d9e5d62734878892179a 127.0.0.1:7003 slots:10923-16383 (5461 slots) master 1 additional replica(s) M: 6935de6f15a45ca0aa0d6c785361e85bb3593585 127.0.0.1:7002 slots:5461-10922 (5462 slots) master 1 additional replica(s) Destination node: M: 608a43079e6f61dfc8200c29fbad1c4cd721b06f 127.0.0.1:7001 slots:0-5460 (5461 slots) master 1 additional replica(s)

添加slave新节点，依旧新新建一个redis实例：

redis-cli连接上新节点shell,输入命令:cluster replicate 对应master的node-id

note:在线添加slave 时，需要dump整个master进程，并传递到slave，再由 slave加载rdb文件到内存，rdb传输过程中Master可能无法提供服务,整个过程消耗大量io,小心操作.

对于负载/数据均匀的情况，可以在线reshard slot来解决,方法与添加新master的reshard一样，只是需要reshard的master节点是老节点.

6. redis集群缩容

6.1 删除一个slave节点：

＃redis-trib.rb del-node 127.0.0.1:7008  'c7ee2fca17cb79fe3c9822ced1d4f6c5e169e378'

6.2 删除一个master节点：

删除master节点之前首先要使用reshard移除master的全部slot,然后再删除当前节点(目前只能把被删除master的slot迁移到一个节点上)

示例：

把刚才新加大7007主节点的数据移到7001

./redis-trib.rb reshard 127.0.0.1:7001

重新分片，吧想要7001节点的slot数量给移动到那个master的ID就行，保证7001的插槽是空的

# ./redis-trib.rb del-node 127.0.0.1:7001 608a43079e6f61dfc8200c29fbad1c4cd721b06f

>>> Removing node 608a43079e6f61dfc8200c29fbad1c4cd721b06f from cluster 127.0.0.1:7001 Connecting to node 127.0.0.1:7001: OK Connecting to node 127.0.0.1:7007: OK Connecting to node 127.0.0.1:7006: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7005: OK Connecting to node 127.0.0.1:7004: OK Connecting to node 127.0.0.1:7003: OK >>> Sending CLUSTER FORGET messages to the cluster... >>> 127.0.0.1:7004 as replica of 127.0.0.1:7007 >>> SHUTDOWN the node.

这样子，7007就代表了7001的master的位置，7004替换成slave

查看每个节点的ID，以及当前运行的的节点

# /export/servers/redis-3.0.5/src/redis-cli -p 7002 cluster nodes

48e6cdd7355b49a63ab9f29fdd2bf7a7738505b5 127.0.0.1:7007 master - 0 1450844733682 8 connected 6074-7476 11172-12937 6935de6f15a45ca0aa0d6c785361e85bb3593585 127.0.0.1:7002 myself,master - 0 0 9 connected 0-3710 5712-6073 7477-10922 12938-14415 584e9035655d0de8dd4047c685363c2e5634e4e0 127.0.0.1:7006 slave 58d18867c2c17a9f9b30d9e5d62734878892179a 0 1450844734684 10 connected b230c9430a636f2421c323bb6cccda1c4dff16b7 127.0.0.1:7005 slave 6935de6f15a45ca0aa0d6c785361e85bb3593585 0 1450844735686 9 connected 82b8684f0b35f0147fed50254853898193fe0336 127.0.0.1:7004 slave 48e6cdd7355b49a63ab9f29fdd2bf7a7738505b5 0 1450844732680 8 connected 58d18867c2c17a9f9b30d9e5d62734878892179a 127.0.0.1:7003 master - 0 1450844736689 10 connected 3711-5711 10923-11171 14416-16383