外包精通--k8s之flannel网络
原创
flannel网络
flannel网络有三种：
1、vxlan（隧道方案）不需要后续操作，pod与pod之间网络是互通的。 2、host-gw（路由方案）只能在二层网络中，不支持跨网络，如需胡同需要添加路由。不推荐使用 3、udp（由于性能较差已经弃用）
Vxlan
vxlan概述
vxlan(virtual Extensible LAN)虚拟可扩展局域网，是一种overlay的网络技术，使用MAC in UDP的方法进
行封装，共50字节的封装报文头。
感兴趣可以上网搜索vxlan
软件版本
相关软件安装就不过多的讲了，可以上网搜索怎么安装。
版本 | 说明 | |
|---|---|---|
操作系统 | CentOS Linux release 7.9.2009 (Core) | |
docker | 20.10.8 | |
kubeadm | v1.18.5 | |
kubectl | v1.18.5 | |
kubelet | v1.18.5 |
[root@VM-3-8-centos ~]# docker version
Client: Docker Engine - Community
Version: 20.10.8
API version: 1.41
Go version: go1.16.6
Git commit: 3967b7d
[root@VM-3-8-centos ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:45:16Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
[root@VM-3-8-centos ~]# 网络配置信息
网卡信息
[root@VM-3-8-centos ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 52:54:00:c6:6c:02 brd ff:ff:ff:ff:ff:ff
inet 172.16.3.8/24 brd 172.16.3.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fec6:6c02/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:15:1b:8d:77 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
[root@VM-3-8-centos ~]#路由信息
[root@VM-3-8-centos ~]# ip route
default via 172.16.3.1 dev eth0
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.8
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
[root@VM-3-8-centos ~]# master节点去除不可以调度
[root@VM-3-8-centos ~]# kubectl taint node vm-3-8-centos node-role.kubernetes.io/master-
node/vm-3-8-centos untainted
[root@VM-3-8-centos ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
vm-3-5-centos Ready <none> 108s v1.18.5
vm-3-8-centos Ready master 19m v1.18.5
[root@VM-3-8-centos ~]# flannel部署
获取flannel部署文件
github地址：https://github.com/flannel-io/flannel
从GitHub上可以找到相关部署说明
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml查看网络模式
可以从yaml文件中看到默认模式就是vxlan模式：
[root@VM-3-8-centos ~]# wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
--2021-08-29 00:00:05-- https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.108.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4813 (4.7K) [text/plain]
Saving to: ‘kube-flannel.yml’
kube-flannel.yml 100%[===========================================================>] 4.70K --.-KB/s in 0s
2021-08-29 00:00:05 (57.4 MB/s) - ‘kube-flannel.yml’ saved [4813/4813]
[root@VM-3-8-centos ~]#
[root@VM-3-8-centos ~]# cat kube-flannel.yml |grep vxlan
"Type": "vxlan"
[root@VM-3-8-centos ~]# 开始部署flannel
通过GitHub或者官方文档，使用kubectl即可部署flannel
[root@VM-3-8-centos ~]# kubectl apply -f kube-flannel.yml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
[root@VM-3-8-centos ~]# 节点状态
[root@VM-3-8-centos ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
vm-3-5-centos Ready <none> 108s v1.18.5
vm-3-8-centos Ready master 19m v1.18.5
[root@VM-3-8-centos ~]# 部署成功网卡信息
[root@VM-3-8-centos ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 52:54:00:c6:6c:02 brd ff:ff:ff:ff:ff:ff
inet 172.16.3.8/24 brd 172.16.3.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fec6:6c02/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:15:1b:8d:77 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
link/ether 96:a0:6b:56:28:5f brd ff:ff:ff:ff:ff:ff
inet 10.244.0.0/32 brd 10.244.0.0 scope global flannel.1
valid_lft forever preferred_lft forever
inet6 fe80::94a0:6bff:fe56:285f/64 scope link
valid_lft forever preferred_lft forever
5: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether de:cd:fb:ea:85:bd brd ff:ff:ff:ff:ff:ff
inet 10.244.0.1/24 brd 10.244.0.255 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::dccd:fbff:feea:85bd/64 scope link
valid_lft forever preferred_lft forever
6: veth8853a7f0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether 92:65:9a:b1:77:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::9065:9aff:feb1:7734/64 scope link
valid_lft forever preferred_lft forever
7: vethcf75cd48@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether 86:37:04:6e:78:fc brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::8437:4ff:fe6e:78fc/64 scope link
valid_lft forever preferred_lft forever
[root@VM-3-8-centos ~]# 部署成功路由信息
[root@VM-3-8-centos ~]# ip route
default via 172.16.3.1 dev eth0
10.244.0.0/24 dev cni0 proto kernel scope link src 10.244.0.1
169.254.0.0/16 dev eth0 scope link metric 1002
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.8
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
[root@VM-3-8-centos ~]# [root@VM-3-8-centos ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.3.1 0.0.0.0 UG 0 0 0 eth0
10.244.0.0 0.0.0.0 255.255.255.0 U 0 0 0 cni0
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
172.16.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
[root@VM-3-8-centos ~]# [root@VM-3-8-centos ~]# arp -n
Address HWtype HWaddress Flags Mask Iface
169.254.0.79 ether fe:ee:59:af:45:0e C eth0
10.244.0.2 ether 22:09:a5:99:3e:e0 C cni0
10.244.0.3 ether 9e:f8:a4:b4:5a:1c C cni0
169.254.0.47 ether fe:ee:59:af:45:0e C eth0
169.254.169.254 ether fe:ee:59:af:45:0e C eth0
169.254.128.12 ether fe:ee:59:af:45:0e C eth0
169.254.0.15 ether fe:ee:59:af:45:0e C eth0
169.254.0.3 ether fe:ee:59:af:45:0e C eth0
169.254.0.2 ether fe:ee:59:af:45:0e C eth0
169.254.128.2 ether fe:ee:59:af:45:0e C eth0
169.254.0.4 ether fe:ee:59:af:45:0e C eth0
169.254.0.81 ether fe:ee:59:af:45:0e C eth0
169.254.0.80 ether fe:ee:59:af:45:0e C eth0
169.254.0.83 ether fe:ee:59:af:45:0e C eth0
169.254.0.82 ether fe:ee:59:af:45:0e C eth0
169.254.0.138 ether fe:ee:59:af:45:0e C eth0
172.16.3.1 ether fe:ee:59:af:45:0e C eth0
169.254.0.55 ether fe:ee:59:af:45:0e C eth0
169.254.0.23 ether fe:ee:59:af:45:0e C eth0
[root@VM-3-8-centos ~]# flannel.1配置信息
通过ip -d link命令查看VTEP设备flannel.1的配置信息： vxlan id 1 local 172.16.3.8 dev eth0 srcport 0 0 dstport 8472
[root@VM-3-8-centos ~]# ip -d link show flannel.1
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/ether 96:a0:6b:56:28:5f brd ff:ff:ff:ff:ff:ff promiscuity 0
vxlan id 1 local 172.16.3.8 dev eth0 srcport 0 0 dstport 8472 nolearning ageing 300 noudpcsum noudp6zerocsumtx noudp6zerocsumrx addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
[root@VM-3-8-centos ~]# 集群配置查看
在集群中到处yaml信息可以看到也是vxlan模式
[root@VM-3-8-centos ~]# kubectl get configmaps kube-flannel-cfg -n kube-system -oyaml
apiVersion: v1
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
kind: ConfigMap
.....
[root@VM-3-8-centos ~]# 查看kube-flannel日志
[root@VM-3-8-centos ~]# kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-66bff467f8-dc6mw 1/1 Running 0 24m
kube-system coredns-66bff467f8-z59fb 1/1 Running 0 24m
kube-system etcd-vm-3-8-centos 1/1 Running 0 24m
kube-system kube-apiserver-vm-3-8-centos 1/1 Running 0 24m
kube-system kube-controller-manager-vm-3-8-centos 1/1 Running 0 24m
kube-system kube-flannel-ds-cjqwc 1/1 Running 0 21m
kube-system kube-proxy-7gfr5 1/1 Running 0 24m
kube-system kube-scheduler-vm-3-8-centos 1/1 Running 0 24m
[root@VM-3-8-centos ~]#
[root@VM-3-8-centos ~]# kubectl logs -f -n kube-system kube-flannel-ds-cjqwc
I0827 14:44:57.981013 1 main.go:520] Determining IP address of default interface
I0827 14:44:57.981444 1 main.go:533] Using interface with name eth0 and address 172.16.3.8
I0827 14:44:57.981472 1 main.go:550] Defaulting external address to interface address (172.16.3.8)
W0827 14:44:57.981496 1 client_config.go:608] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0827 14:44:58.174432 1 kube.go:116] Waiting 10m0s for node controller to sync
I0827 14:44:58.174816 1 kube.go:299] Starting kube subnet manager
I0827 14:44:59.174795 1 kube.go:123] Node controller sync successful
I0827 14:44:59.174839 1 main.go:254] Created subnet manager: Kubernetes Subnet Manager - vm-3-8-centos
I0827 14:44:59.174846 1 main.go:257] Installing signal handlers
I0827 14:44:59.175023 1 main.go:392] Found network config - Backend type: vxlan
I0827 14:44:59.175106 1 vxlan.go:123] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
# `这是手动修改，通过日志可以看到是vxlan模式的` Found network config - Backend type: vxlan
I0827 14:44:59.217943 1 main.go:357] Current network or subnet (10.244.0.0/16, 10.244.0.0/24) is not equal to previous one (0.0.0.0/0, 0.0.0.0/0), trying to recycle old iptables rules
I0827 14:44:59.379104 1 iptables.go:172] Deleting iptables rule: -s 0.0.0.0/0 -d 0.0.0.0/0 -j RETURN
I0827 14:44:59.380080 1 iptables.go:172] Deleting iptables rule: -s 0.0.0.0/0 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I0827 14:44:59.380948 1 iptables.go:172] Deleting iptables rule: ! -s 0.0.0.0/0 -d 0.0.0.0/0 -j RETURN
I0827 14:44:59.381830 1 iptables.go:172] Deleting iptables rule: ! -s 0.0.0.0/0 -d 0.0.0.0/0 -j MASQUERADE --random-fully
I0827 14:44:59.473380 1 main.go:307] Setting up masking rules
I0827 14:44:59.474319 1 main.go:315] Changing default FORWARD chain policy to ACCEPT
I0827 14:44:59.474402 1 main.go:323] Wrote subnet file to /run/flannel/subnet.env
I0827 14:44:59.474410 1 main.go:327] Running backend.
I0827 14:44:59.474423 1 main.go:345] Waiting for all goroutines to exit
I0827 14:44:59.474450 1 vxlan_network.go:59] watching for new subnet leases
I0827 14:44:59.476439 1 iptables.go:148] Some iptables rules are missing; deleting and recreating rules
I0827 14:44:59.476453 1 iptables.go:172] Deleting iptables rule: -s 10.244.0.0/16 -j ACCEPT
I0827 14:44:59.477284 1 iptables.go:148] Some iptables rules are missing; deleting and recreating rules
I0827 14:44:59.477297 1 iptables.go:172] Deleting iptables rule: -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN
I0827 14:44:59.478102 1 iptables.go:172] Deleting iptables rule: -d 10.244.0.0/16 -j ACCEPT
I0827 14:44:59.479002 1 iptables.go:172] Deleting iptables rule: -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I0827 14:44:59.480003 1 iptables.go:172] Deleting iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/24 -j RETURN
I0827 14:44:59.481069 1 iptables.go:172] Deleting iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE --random-fully
I0827 14:44:59.481072 1 iptables.go:160] Adding iptables rule: -s 10.244.0.0/16 -j ACCEPT
I0827 14:44:59.574901 1 iptables.go:160] Adding iptables rule: -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN
I0827 14:44:59.575108 1 iptables.go:160] Adding iptables rule: -d 10.244.0.0/16 -j ACCEPT
I0827 14:44:59.676528 1 iptables.go:160] Adding iptables rule: -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I0827 14:44:59.678443 1 iptables.go:160] Adding iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/24 -j RETURN
I0827 14:44:59.680182 1 iptables.go:160] Adding iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE --random-fully[root@VM-3-8-centos ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.3.1 0.0.0.0 UG 0 0 0 eth0
10.244.0.0 0.0.0.0 255.255.255.0 U 0 0 0 cni0
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
172.16.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
[root@VM-3-8-centos ~]#
[root@VM-3-8-centos ~]#
[root@VM-3-8-centos ~]# arp -n
Address HWtype HWaddress Flags Mask Iface
169.254.0.79 ether fe:ee:59:af:45:0e C eth0
10.244.0.2 ether 22:09:a5:99:3e:e0 C cni0
10.244.0.3 ether 9e:f8:a4:b4:5a:1c C cni0
169.254.0.47 ether fe:ee:59:af:45:0e C eth0
169.254.169.254 ether fe:ee:59:af:45:0e C eth0
169.254.128.12 ether fe:ee:59:af:45:0e C eth0
169.254.0.15 ether fe:ee:59:af:45:0e C eth0
169.254.0.3 ether fe:ee:59:af:45:0e C eth0
169.254.0.2 ether fe:ee:59:af:45:0e C eth0
169.254.128.2 ether fe:ee:59:af:45:0e C eth0
169.254.0.4 ether fe:ee:59:af:45:0e C eth0
169.254.0.81 ether fe:ee:59:af:45:0e C eth0
169.254.0.80 ether fe:ee:59:af:45:0e C eth0
169.254.0.83 ether fe:ee:59:af:45:0e C eth0
169.254.0.82 ether fe:ee:59:af:45:0e C eth0
169.254.0.138 ether fe:ee:59:af:45:0e C eth0
172.16.3.1 ether fe:ee:59:af:45:0e C eth0
169.254.0.55 ether fe:ee:59:af:45:0e C eth0
169.254.0.23 ether fe:ee:59:af:45:0e C eth0
[root@VM-3-8-centos ~]# 查看桥接
[root@VM-3-8-centos ~]# brctl show
bridge name bridge id STP enabled interfaces
cni0 8000.decdfbea85bd no veth8853a7f0
vethcf75cd48
docker0 8000.0242151b8d77 no
[root@VM-3-8-centos ~]#创建pod
[root@VM-3-8-centos ~]# kubectl create deployment web --image=nginx
deployment.apps/web created
[root@VM-3-8-centos ~]# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
[root@VM-3-8-centos ~]#[root@VM-3-8-centos ~]# kubectl get pod -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default nginx-f89759699-xq7xg 1/1 Running 0 15m 10.244.0.6 vm-3-8-centos <none> <none>
default web-5dcb957ccc-prg7j 1/1 Running 0 6m50s 10.244.1.2 vm-3-5-centos <none> <none>
kube-system coredns-66bff467f8-86rsq 1/1 Running 1 26m 10.244.0.5 vm-3-8-centos <none> <none>
kube-system coredns-66bff467f8-qwjlv 1/1 Running 1 26m 10.244.0.4 vm-3-8-centos <none> <none>
kube-system etcd-vm-3-8-centos 1/1 Running 1 26m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-apiserver-vm-3-8-centos 1/1 Running 1 26m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-controller-manager-vm-3-8-centos 1/1 Running 1 26m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-flannel-ds-fhww2 1/1 Running 0 19s 172.16.3.5 vm-3-5-centos <none> <none>
kube-system kube-flannel-ds-ln4sw 1/1 Running 0 19s 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-proxy-5sbmg 1/1 Running 0 8m33s 172.16.3.5 vm-3-5-centos <none> <none>
kube-system kube-proxy-7dsgg 1/1 Running 1 26m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-scheduler-vm-3-8-centos 1/1 Running 1 26m 172.16.3.8 vm-3-8-centos <none> <none>
[root@VM-3-8-centos ~]# 网络连接测试
[root@VM-3-8-centos ~]# ping 10.244.0.6 -c 4
PING 10.244.0.6 (10.244.0.6) 56(84) bytes of data.
64 bytes from 10.244.0.6: icmp_seq=1 ttl=64 time=0.073 ms
64 bytes from 10.244.0.6: icmp_seq=2 ttl=64 time=0.042 ms
64 bytes from 10.244.0.6: icmp_seq=3 ttl=64 time=0.038 ms
64 bytes from 10.244.0.6: icmp_seq=4 ttl=64 time=0.036 ms
--- 10.244.0.6 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.036/0.047/0.073/0.015 ms
[root@VM-3-8-centos ~]# ping 10.244.1.2 -c 2
PING 10.244.1.2 (10.244.1.2) 56(84) bytes of data.
64 bytes from 10.244.1.2: icmp_seq=1 ttl=63 time=0.293 ms
64 bytes from 10.244.1.2: icmp_seq=2 ttl=63 time=0.259 ms
--- 10.244.1.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.259/0.276/0.293/0.017 ms
[root@VM-3-8-centos ~]# pod连接测试
可以看到nginx在节点①上，通过curl可以访问web页面
[root@VM-3-8-centos ~]# curl -v 10.244.0.6
* About to connect() to 10.244.0.6 port 80 (#0)
* Trying 10.244.0.6...
* Connected to 10.244.0.6 (10.244.0.6) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.244.0.6
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.21.1
< Date: Fri, 27 Aug 2021 16:02:47 GMT
< Content-Type: text/html
< Content-Length: 612
< Last-Modified: Tue, 06 Jul 2021 14:59:17 GMT
< Connection: keep-alive
< ETag: "60e46fc5-264"
< Accept-Ranges: bytes
<
......
* Connection #0 to host 10.244.0.6 left intact
[root@VM-3-8-centos ~]# [root@VM-3-8-centos ~]# curl -v 10.244.1.2
* About to connect() to 10.244.1.2 port 80 (#0)
* Trying 10.244.1.2...
* Connected to 10.244.1.2 (10.244.1.2) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.244.1.2
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.21.1
< Date: Fri, 27 Aug 2021 16:02:55 GMT
< Content-Type: text/html
< Content-Length: 612
< Last-Modified: Tue, 06 Jul 2021 14:59:17 GMT
< Connection: keep-alive
< ETag: "60e46fc5-264"
< Accept-Ranges: bytes
<
.....
* Connection #0 to host 10.244.1.2 left intact
[root@VM-3-8-centos ~]#删除flannel
[root@VM-3-8-centos ~]# kubectl delete -f kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged deleted
clusterrole.rbac.authorization.k8s.io/flannel deleted
clusterrolebinding.rbac.authorization.k8s.io/flannel deleted
serviceaccount/flannel deleted
configmap/kube-flannel-cfg deleted
daemonset.apps/kube-flannel-ds deleted
[root@VM-3-8-centos ~]# 删除网卡配置
[root@knode01 ~]# ifconfig cni0 down
[root@knode01 ~]# ifconfig docker0 down
[root@knode01 ~]# ifconfig flannel.1 down
[root@knode01 ~]# ip link delete cni0
[root@knode01 ~]# ip link delete flannel.1 Host-gw
修改配置文件为host-gw
[root@VM-3-8-centos ~]# cat kube-flannel.yml|grep Type
"Type": "host-gw"
[root@VM-3-8-centos ~]# 开始部署flannel
[root@VM-3-8-centos ~]# kubectl apply -f kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
[root@VM-3-8-centos ~]# 网卡信息
[root@VM-3-8-centos ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 52:54:00:c6:6c:02 brd ff:ff:ff:ff:ff:ff
inet 172.16.3.8/24 brd 172.16.3.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fec6:6c02/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:02:a5:20:52 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
4: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 2a:c7:1d:9c:a1:c0 brd ff:ff:ff:ff:ff:ff
inet 10.244.0.1/24 brd 10.244.0.255 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::28c7:1dff:fe9c:a1c0/64 scope link
valid_lft forever preferred_lft forever
5: veth17eaf140@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni0 state UP group default
link/ether 06:8e:ff:87:cc:18 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::48e:ffff:fe87:cc18/64 scope link
valid_lft forever preferred_lft forever
6: veth37b3cbaa@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni0 state UP group default
link/ether 1e:1c:25:d9:92:29 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::1c1c:25ff:fed9:9229/64 scope link
valid_lft forever preferred_lft forever
[root@VM-3-8-centos ~]# 路由信息
[root@VM-3-8-centos ~]# ip route
default via 172.16.3.1 dev eth0
10.244.0.0/24 dev cni0 proto kernel scope link src 10.244.0.1
169.254.0.0/16 dev eth0 scope link metric 1002
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.8
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
[root@VM-3-8-centos ~]# [root@VM-3-8-centos ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.3.1 0.0.0.0 UG 0 0 0 eth0
10.244.0.0 0.0.0.0 255.255.255.0 U 0 0 0 cni0
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
172.16.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
[root@VM-3-8-centos ~]# 创建pod测试
[root@VM-3-8-centos ~]# kubectl create deployment web --image=nginx
deployment.apps/web created
[root@VM-3-8-centos ~]# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
[root@VM-3-8-centos ~]#[root@VM-3-8-centos ~]# kubectl get pod -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default nginx-f89759699-xq7xg 1/1 Running 0 10m 10.244.0.6 vm-3-8-centos <none> <none>
default web-5dcb957ccc-prg7j 1/1 Running 0 2m9s 10.244.1.2 vm-3-5-centos <none> <none>
kube-system coredns-66bff467f8-86rsq 1/1 Running 1 21m 10.244.0.5 vm-3-8-centos <none> <none>
kube-system coredns-66bff467f8-qwjlv 1/1 Running 1 21m 10.244.0.4 vm-3-8-centos <none> <none>
kube-system etcd-vm-3-8-centos 1/1 Running 1 21m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-apiserver-vm-3-8-centos 1/1 Running 1 21m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-controller-manager-vm-3-8-centos 1/1 Running 1 21m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-flannel-ds-sl47n 1/1 Running 0 14m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-flannel-ds-xzdk2 1/1 Running 0 3m52s 172.16.3.5 vm-3-5-centos <none> <none>
kube-system kube-proxy-5sbmg 1/1 Running 0 3m52s 172.16.3.5 vm-3-5-centos <none> <none>
kube-system kube-proxy-7dsgg 1/1 Running 1 21m 172.16.3.8 vm-3-8-centos <none> <none>
kube-system kube-scheduler-vm-3-8-centos 1/1 Running 1 21m 172.16.3.8 vm-3-8-centos <none> <none>
[root@VM-3-8-centos ~]#网络连接测试
[root@VM-3-5-centos ~]# ping 10.244.0.6 -c 4
PING 10.244.1.2 (10.244.1.2) 56(84) bytes of data.
64 bytes from 10.244.1.2: icmp_seq=1 ttl=64 time=0.041 ms
64 bytes from 10.244.1.2: icmp_seq=2 ttl=64 time=0.042 ms
64 bytes from 10.244.1.2: icmp_seq=3 ttl=64 time=0.043 ms
64 bytes from 10.244.1.2: icmp_seq=4 ttl=64 time=0.044 ms
--- 10.244.1.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.041/0.042/0.044/0.006 ms
[root@VM-3-5-centos ~]# [root@VM-3-8-centos ~]# ping 10.244.1.2
PING 10.244.1.2 (10.244.1.2) 56(84) bytes of data.
^C
--- 10.244.1.2 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms
[root@VM-3-8-centos ~]# [root@VM-3-8-centos ~]# curl -v 10.244.0.6
* About to connect() to 10.244.0.6 port 80 (#0)
* Trying 10.244.0.6...
* Connected to 10.244.0.6 (10.244.0.6) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.244.0.6
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.21.1
< Date: Fri, 27 Aug 2021 15:57:47 GMT
< Content-Type: text/html
< Content-Length: 612
< Last-Modified: Tue, 06 Jul 2021 14:59:17 GMT
< Connection: keep-alive
< ETag: "60e46fc5-264"
< Accept-Ranges: bytes
<
......
* Connection #0 to host 10.244.0.6 left intact
[root@VM-3-8-centos ~]# [root@VM-3-8-centos ~]# curl -v 10.244.1.2
* About to connect() to 10.244.1.2 port 80 (#0)
* Trying 10.244.1.2...
^C
[root@VM-3-8-centos ~]# [root@VM-3-5-centos ~]# curl -v 10.244.1.2 -c 4
* About to connect() to 10.244.1.2 port 80 (#0)
* Trying 10.244.1.2...
* Connected to 10.244.1.2 (10.244.1.2) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.244.1.2
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.21.1
< Date: Fri, 27 Aug 2021 15:59:58 GMT
< Content-Type: text/html
< Content-Length: 612
< Last-Modified: Tue, 06 Jul 2021 14:59:17 GMT
< Connection: keep-alive
< ETag: "60e46fc5-264"
< Accept-Ranges: bytes
<
......
* Connection #0 to host 10.244.1.2 left intact
[root@VM-3-5-centos ~]# 添加路由
为什么要添加路由你呢？因为在host-gw网络模式中，只有在宿主机下才能访问pod。跨主机是访问不同pod的。
根据实际的网络信息进行添加
[root@VM-3-5-centos ~]# route add -net 10.244.0.0 netmask 255.255.0.0 dev flannel.1检测网络
[root@VM-3-8-centos ~]# ping 10.244.0.6 -c 4
PING 10.244.0.6 (10.244.0.6) 56(84) bytes of data.
64 bytes from 10.244.0.6: icmp_seq=1 ttl=64 time=0.073 ms
64 bytes from 10.244.0.6: icmp_seq=2 ttl=64 time=0.042 ms
64 bytes from 10.244.0.6: icmp_seq=3 ttl=64 time=0.038 ms
64 bytes from 10.244.0.6: icmp_seq=4 ttl=64 time=0.036 ms
--- 10.244.0.6 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.036/0.047/0.073/0.015 ms
[root@VM-3-8-centos ~]# ping 10.244.1.2 -c 2
PING 10.244.1.2 (10.244.1.2) 56(84) bytes of data.
64 bytes from 10.244.1.2: icmp_seq=1 ttl=63 time=0.293 ms
64 bytes from 10.244.1.2: icmp_seq=2 ttl=63 time=0.259 ms
--- 10.244.1.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.259/0.276/0.293/0.017 ms
[root@VM-3-8-centos ~]# 原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。
如有侵权，请联系 cloudcommunity@tencent.com 删除。
原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。
如有侵权，请联系 cloudcommunity@tencent.com 删除。
评论
登录后参与评论
推荐阅读