k8s集群部署四（部署Flannel网络） 顶

Flannel网络是在以太网的基础上再封装的一个包含容器IP地址的虚拟网络。

在master节点上建一个文件夹

mkdir flannel

cd flannel

下载安装包

wget https://github.com/coreos/flannel/releases/download/v0.9.1/flannel-v0.9.1-linux-amd64.tar.gz

解压

tar -xzvf flannel-v0.9.1-linux-amd64.tar.gz

将解压后的两个文件flanneld和mk-docker-opts.sh分别拷贝到node节点上

因为我的hosts文件映射为

172.18.98.46 host1 172.18.98.47 host2 172.18.98.48 master

scp flanneld mk-docker-opts.sh root@host1:/opt/kubernetes/bin/

scp flanneld mk-docker-opts.sh root@host2:/opt/kubernetes/bin/

在host1和host2中分别添加网段

cd /opt/kubernetes/ssl

# etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379" set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'

查看

# etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379" get /coreos.com/network/config { "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}

cd /opt/kubernetes/cfg

touch flanneld

vim flanneld

内容如下

FLANNEL_OPTIONS="--etcd-endpoints=https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379 -etcd-cafile=/opt/kubernetes/ssl/ca.pem -etcd-certfile=/opt/kubernetes/ssl/server.pem -etcd-keyfile=/opt/kubernetes/ssl/server-key.pem"

cd /usr/lib/systemd/system

touch flanneld.service

vim flanneld.service

内容如下

[Unit] Description=Flanneld overlay address etcd agent After=network-online.target network.target Before=docker.service

[Service] Type=notify EnvironmentFile=/opt/kubernetes/cfg/flanneld ExecStart=/opt/kubernetes/bin/flanneld --ip-masq $FLANNEL_OPTIONS ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env Restart=on-failure

[Install] WantedBy=multi-user.target

启动flanneld.service

service flanneld start

成功启动，查看进程

# ps -ef | grep flanneld root 24305 1 0 14:28 ? 00:00:01 /opt/kubernetes/bin/flanneld --ip-masq --etcd-endpoints=https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379 -etcd-cafile=/opt/kubernetes/ssl/ca.pem -etcd-certfile=/opt/kubernetes/ssl/server.pem -etcd-keyfile=/opt/kubernetes/ssl/server-key.pem

通过ifconfig,我们可以看到多了一个flannel.1的虚拟网卡

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 172.17.27.0 netmask 255.255.255.255 broadcast 0.0.0.0 ether 8a:00:81:c6:2a:a1 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 11 overruns 0 carrier 0 collisions 0

查看subnet.env文件

cd /run/flannel/

# cat subnet.env DOCKER_OPT_BIP="--bip=172.17.27.1/24" DOCKER_OPT_IPMASQ="--ip-masq=false" DOCKER_OPT_MTU="--mtu=1450" DOCKER_NETWORK_OPTIONS=" --bip=172.17.27.1/24 --ip-masq=false --mtu=1450"

这里面就是分配了一个子网。

让docker使用该网络，修改docker.service

cd /usr/lib/systemd/system

vim docker.service

修改内容

[Service] Type=notify EnvironmentFile=/run/flannel/subnet.env # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS

红色部分为修改内容

重启docker

systemctl daemon-reload & systemctl restart docker

再次执行ifconfig，可以看到flannel和docker的网桥在同一个网段，表示开始生效

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 172.17.27.1 netmask 255.255.255.0 broadcast 172.17.27.255 ether 02:42:ff:c9:b9:9a txqueuelen 0 (Ethernet) RX packets 9430218 bytes 10206182292 (9.5 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7696548 bytes 2199505782 (2.0 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 172.17.27.0 netmask 255.255.255.255 broadcast 0.0.0.0 ether 8a:00:81:c6:2a:a1 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 11 overruns 0 carrier 0 collisions 0

在两台node都装好的情况下，可以互相ping一下对方的flannel的网关

比如我在host2上ping host1的flannel网关

# ping 172.17.27.1 PING 172.17.27.1 (172.17.27.1) 56(84) bytes of data. 64 bytes from 172.17.27.1: icmp_seq=1 ttl=64 time=0.440 ms 64 bytes from 172.17.27.1: icmp_seq=2 ttl=64 time=0.379 ms 64 bytes from 172.17.27.1: icmp_seq=3 ttl=64 time=0.333 ms 64 bytes from 172.17.27.1: icmp_seq=4 ttl=64 time=0.363 ms 64 bytes from 172.17.27.1: icmp_seq=5 ttl=64 time=0.377 ms

完全没有问题

查看所有node的flannel网段（在node节点上，任意节点）

cd /opt/kubernetes/ssl

# etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379" ls /coreos.com/network/subnets /coreos.com/network/subnets/172.17.27.0-24 /coreos.com/network/subnets/172.17.94.0-24

这个是我的两个node节点的网段。

获取某一个节点的key

# etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379" get /coreos.com/network/subnets/172.17.27.0-24 {"PublicIP":"172.18.98.46","BackendType":"vxlan","BackendData":{"VtepMAC":"8a:00:81:c6:2a:a1"}}

查看路由

# route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default gateway 0.0.0.0 UG 0 0 0 eth0 link-local 0.0.0.0 255.255.0.0 U 1002 0 0 eth0 172.17.27.0 172.17.27.0 255.255.255.0 UG 0 0 0 flannel.1 172.17.94.0 0.0.0.0 255.255.255.0 U 0 0 0 docker0 172.18.96.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0