//
MongoDB运维与开发(六)---MongoDB集群(5)
//
关于MongoDB的集群运维,之前的文章已经说了很多内容了,这块儿知识点比较多,由于是每天抽空写文章,所以每天能说到的点有限,慢慢一点一点搞懂它,今天我们来看MongDB集群里面的分片相关知识。
关于分片和副本集的概念,之前的文章中也讲过,上一篇文章中讲了MongoS和ConfigDB的配置方法,今天来看如何在MongoS上添加Mongod分片。
01
初始化分片
通常情况下,我们配置好了mongos和configDB之后,需要在Mongos上面配置分片节点,配置分片节点的之前,需要有几项准备工作:
1、启动分片节点副本集
2、在mongos的配置文件中写清除configDB的地址
3、保证mongos可以同时访问configDB和分片节点
然后,我们通过下面的命令来配置初始化添加一个分片节点副本集:
use admin
db.runCommand({addShard:"sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020"})
可以看到,我们添加了一个名字叫做sharding_yeyz的副本集,它作为Mongos的分片加入到这个集群中,它包含27018、27019、27020三个mongod实例,架构图如下:
02
在将要分片的集合上面创建索引
mongos> use test
switched to db test
mongos> db.test0.ensureIndex({_id:"hashed"})
{
"raw" : {
"sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020" : {
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1
}
},
"ok" : 1,
"operationTime" : Timestamp(1605690613, 4),
"$clusterTime" : {
"clusterTime" : Timestamp(1605690613, 4),
"signature" : {
"hash" : BinData(0,"9ThfDRTC/X/GSuLbe8E7TjXLP2I="),
"keyId" : NumberLong("6894922308364795934")
}
}
}
如上面的命令,我们在test库下面的test0集合上的_id字段上创建hash索引,也就是:
db.test0.ensureIndex({_id:"hashed"})
03
对db和collection启用分片
首先我们通过下面的命令,对test库启用分片:
mongos> sh.enableSharding("test")
{
"ok" : 1,
"operationTime" : Timestamp(1605690781, 3),
"$clusterTime" : {
"clusterTime" : Timestamp(1605690781, 3),
"signature" : {
"hash" : BinData(0,"YrrrniMbOp8Vfjkf8cYulX+n9oA="),
"keyId" : NumberLong("6894922308364795934")
}
}
}
可以看到,命令sh.enableSharding对库test启用了分片。
然后对test0这个collection启用分片:
mongos> sh.shardCollection("test.test0",{_id:"hashed"})
{
"collectionsharded" : "test.test0",
"collectionUUID" : UUID("a1bafe79-c74e-4454-ae98-ce1b175cda2c"),
"ok" : 1,
"operationTime" : Timestamp(1605690845, 10),
"$clusterTime" : {
"clusterTime" : Timestamp(1605690845, 10),
"signature" : {
"hash" : BinData(0,"PIttL1zPmNcjjGZ1BpBIZVrhT6I="),
"keyId" : NumberLong("6894922308364795934")
}
}
}
04
插入数据,观察分片状态
我们循环插入number=1、2、3~1000的数字,并观察当前分片状态如下:
### 循环程序,插入i=1、2、3、4...1000到test.test0这个集合
mongos> for (var i=1 ;i<=1000 ; i++){ db.test1.insert({"number":i})}
WriteResult({ "nInserted" : 1 })
### 使用printShardingStatus查看分片状态
mongos> db.printShardingStatus()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5fafaf4f5785d9965548f687")
}
shards: # 涉及的分片只有sharding_yeyz
{ "_id" : "sharding_yeyz", "host" : "sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020", "state" : 1 }
active mongoses:
"4.0.6" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 1
Last reported error: Could not find host matching read preference { mode: "primary" } for set sharding_yeyz
Time of Reported error: Wed Nov 18 2020 17:08:14 GMT+0800 (CST)
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
sharding_yeyz 1
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : sharding_yeyz Timestamp(1, 0)
{ "_id" : "test", "primary" : "sharding_yeyz", "partitioned" : true, "version" : { "uuid" : UUID("f1d54510-2675-45e1-b61b-2eb0fe015f2d"), "lastMod" : 1 } }
test.test0
shard key: { "_id" : "hashed" }
unique: false
balancing: true
chunks: # 两个数据chunk都在sharding_yeyz上
sharding_yeyz 2
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong(0) } on : sharding_yeyz Timestamp(1, 0)
{ "_id" : NumberLong(0) } -->> { "_id" : { "$maxKey" : 1 } } on : sharding_yeyz Timestamp(1, 1)
从分片状态可以看到,当前只存在一个分片sharding_yeyz,所有的数据都在该分片上,然后我们查询一下数据:
sharding_yeyz:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99a"), "number" : 1 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99b"), "number" : 2 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99c"), "number" : 3 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99d"), "number" : 4 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99e"), "number" : 5 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99f"), "number" : 6 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a0"), "number" : 7 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a1"), "number" : 8 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a2"), "number" : 9 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a3"), "number" : 10 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a4"), "number" : 11 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a5"), "number" : 12 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a6"), "number" : 13 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a7"), "number" : 14 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a8"), "number" : 15 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a9"), "number" : 16 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9aa"), "number" : 17 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ab"), "number" : 18 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ac"), "number" : 19 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ad"), "number" : 20 }
Type "it" for more
sharding_yeyz:PRIMARY> db.test0.count()
1000
可以看到所有数据都在,数据总条数是1000条,和我们设想的结果一致。
05
加入一个新的分片
此时我们加入一个新的分片副本集,名字叫做sharding_yeyz1,它由27024、27025、27026三个mongod实例组成,如下:
db.runCommand({addShard:"sharding_yeyz1/127.0.0.1:27024,127.0.0.1:27025,127.0.0.1:27026"})
{
"shardAdded" : "sharding_yeyz1",
"ok" : 1,
"operationTime" : Timestamp(1605697822, 2),
"$clusterTime" : {
"clusterTime" : Timestamp(1605697822, 3),
"signature" : {
"hash" : BinData(0,"wpspZ+ILVa7R5Zybhks+3da9N4g="),
"keyId" : NumberLong("6894922308364795934")
}
}
}
数据的架构变成了:
此时我们重新对test库下面的test1集合启用分片,并插入1~1000的数据,如下:
启用分片
sh.shardCollection("test.test1",{_id:"hashed"})
插入数据
for (var i=1 ;i<=1000 ; i++){ db.test1.insert({"number":i})}
来看sharding_yeyz和sharding_yeyz1上test.test1这个集合的查询结果:
sharding_yeyz:PRIMARY> db.test1.find()
{ "_id" : ObjectId("5fb50645eebfcce3b111e552"), "number" : 1 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e553"), "number" : 2 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e554"), "number" : 3 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e555"), "number" : 4 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e556"), "number" : 5 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e557"), "number" : 6 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55b"), "number" : 10 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55e"), "number" : 13 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e563"), "number" : 18 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e565"), "number" : 20 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e566"), "number" : 21 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e568"), "number" : 23 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e569"), "number" : 24 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56a"), "number" : 25 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56b"), "number" : 26 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56d"), "number" : 28 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56e"), "number" : 29 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e573"), "number" : 34 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e576"), "number" : 37 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e577"), "number" : 38 }
Type "it" for more
sharding_yeyz:PRIMARY> db.test1.count()
519
--------------------
sharding_yeyz1:PRIMARY> db.test1.find()
{ "_id" : ObjectId("5fb50645eebfcce3b111e558"), "number" : 7 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e559"), "number" : 8 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55a"), "number" : 9 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55c"), "number" : 11 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55d"), "number" : 12 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55f"), "number" : 14 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e560"), "number" : 15 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e561"), "number" : 16 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e562"), "number" : 17 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e564"), "number" : 19 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e567"), "number" : 22 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56c"), "number" : 27 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56f"), "number" : 30 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e570"), "number" : 31 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e571"), "number" : 32 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e572"), "number" : 33 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e574"), "number" : 35 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e575"), "number" : 36 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e57a"), "number" : 41 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e57b"), "number" : 42 }
Type "it" for more
sharding_yeyz1:PRIMARY> db.test1.count()
481
可以看到,sharding_yeyz保存有519条数据,而sharding_yeyz1保存有481条数据,并不是严格的按照50%的比例分配的,而且number的序号,也不是严格按照奇数偶数来分的。
还有一个问题?已经分配了的test0集合,1000条记录原本都在sharding_yeyz这个分片上,加入了sharding_yeyz1,这1000条记录会不会重新分配到第二个分片上呢?我们来看实际结果:
------------原始分片------------
sharding_yeyz:PRIMARY> db.test0.count()
492
sharding_yeyz:PRIMARY>
sharding_yeyz:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99b"), "number" : 2 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99d"), "number" : 4 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99f"), "number" : 6 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a1"), "number" : 8 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a2"), "number" : 9 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a3"), "number" : 10 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a4"), "number" : 11 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a5"), "number" : 12 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a6"), "number" : 13 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a7"), "number" : 14 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ae"), "number" : 21 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b0"), "number" : 23 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b2"), "number" : 25 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b3"), "number" : 26 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b4"), "number" : 27 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b5"), "number" : 28 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b6"), "number" : 29 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b7"), "number" : 30 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b9"), "number" : 32 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bd"), "number" : 36 }
Type "it" for more
----------------新增分片------------
sharding_yeyz1:PRIMARY> db.test0.count()
508
sharding_yeyz1:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99a"), "number" : 1 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99c"), "number" : 3 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99e"), "number" : 5 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a0"), "number" : 7 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a8"), "number" : 15 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a9"), "number" : 16 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9aa"), "number" : 17 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ab"), "number" : 18 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ac"), "number" : 19 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ad"), "number" : 20 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9af"), "number" : 22 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b1"), "number" : 24 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b8"), "number" : 31 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ba"), "number" : 33 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bb"), "number" : 34 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bc"), "number" : 35 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9be"), "number" : 37 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bf"), "number" : 38 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9c1"), "number" : 40 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9c4"), "number" : 43 }
从结果中不难看出来,原来test0这个集合上面的1000条记录,也都从分片1上相对均匀的分配到了分片2上面,而分片1上的数据进行了清理,不再是之前的1000条。
到这里,大家对MongoDB的分片应该有了如下认识:
1、分片是自动的
2、分片不是绝对均匀的
后续的文章,我们会说明分片受哪些因素的影响。
今天就到这里吧。