Kubernetes（k8s）中Pod资源的健康检查
1、Pod的健康检查，也叫做探针，探针的种类有两种。
答：1）、livenessProbe，健康状态检查，周期性检查服务是否存活，检查结果失败，将重启容器。 2）、readinessProbe，可用性检查，周期性检查服务是否可用，不可用将从service的endpoints中移除。
2、探针的检测方法。
答：1）、exec，执行一段命令。 2）、httpGet，检测某个http请求的返回状态码。 3）、tcpSocket，测试某个端口是否能够连接。
3、创建exec的探针检测方法。
1 [root@k8s-master health]# vim pod_nginx_exec.yaml 1 apiVersion: v1
2 kind: Pod
3 metadata:
4 name: exec
5 spec:
6 containers:
7 - name: nginx
8 image: 192.168.110.133:5000/nginx:1.13
9 ports:
10 - containerPort: 80
11 args:
12 - /bin/sh
13 - -c
14 - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
15 livenessProbe:
16 exec:
17 command:
18 - cat
19 - /tmp/healthy
20 initialDelaySeconds: 5
21 periodSeconds: 5
22 开始创建这个Pod，操作如下所示：
1 [root@k8s-master health]# kubectl create -f pod_nginx_exec.yaml
2 pod "exec" created
3 [root@k8s-master health]# kubectl get all -o wide
4 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
5 rc/mysql 1 1 1 1h mysql 192.168.110.133:5000/mysql:5.7.30 app=mysql
6 rc/myweb 1 1 1 1h myweb 192.168.110.133:5000/tomcat:latest app=myweb
7
8 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
9 svc/kubernetes 10.254.0.1 <none> 443/TCP 13d <none>
10 svc/mysql 10.254.207.238 <none> 3306/TCP 1h app=mysql
11 svc/myweb 10.254.29.22 <nodes> 8080:30008/TCP 1h app=myweb
12
13 NAME READY STATUS RESTARTS AGE IP NODE
14 po/busybox2 1/1 Running 1 35m 172.16.16.3 k8s-master
15 po/exec 1/1 Running 0 15s 172.16.59.4 k8s-node2
16 po/mysql-lmx4s 1/1 Running 0 1h 172.16.59.2 k8s-node2
17 po/myweb-hsdwn 1/1 Running 0 1h 172.16.32.3 k8s-node3
18 [root@k8s-master health]# 可用通过详细查看Pod的启动状况。
1 [root@k8s-master health]# kubectl describe pod exec
2 Name: exec
3 Namespace: default
4 Node: k8s-node2/192.168.110.134
5 Start Time: Thu, 18 Jun 2020 16:36:16 +0800
6 Labels: <none>
7 Status: Running
8 IP: 172.16.59.4
9 Controllers: <none>
10 Containers:
11 nginx:
12 Container ID: docker://54e08c980bd1ad05c756de4de50b0f61c448a01b7ffcda2b97dc8b5c3a6e2749
13 Image: 192.168.110.133:5000/nginx:1.13
14 Image ID: docker-pullable://192.168.110.133:5000/nginx@sha256:e4f0474a75c510f40b37b6b7dc2516241ffa8bde5a442bde3d372c9519c84d90
15 Port: 80/TCP
16 Args:
17 /bin/sh
18 -c
19 touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
20 State: Running
21 Started: Thu, 18 Jun 2020 16:37:32 +0800
22 Last State: Terminated
23 Reason: Error
24 Exit Code: 137
25 Started: Thu, 18 Jun 2020 16:36:18 +0800
26 Finished: Thu, 18 Jun 2020 16:37:32 +0800
27 Ready: True
28 Restart Count: 1
29 Liveness: exec [cat /tmp/healthy] delay=5s timeout=1s period=5s #success=1 #failure=3
30 Volume Mounts: <none>
31 Environment Variables: <none>
32 Conditions:
33 Type Status
34 Initialized True
35 Ready True
36 PodScheduled True
37 No volumes.
38 QoS Class: BestEffort
39 Tolerations: <none>
40 Events:
41 FirstSeen LastSeen Count From SubObjectPath Type Reason Message
42 --------- -------- ----- ---- ------------- -------- ------ -------
43 1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned exec to k8s-node2
44 1m 1m 1 {kubelet k8s-node2} spec.containers{nginx} Normal Created Created container with docker id ff079949b2cb; Security:[seccomp=unconfined]
45 1m 1m 1 {kubelet k8s-node2} spec.containers{nginx} Normal Started Started container with docker id ff079949b2cb
46 1m 1m 3 {kubelet k8s-node2} spec.containers{nginx} Warning Unhealthy Liveness probe failed: cat: /tmp/healthy: No such file or directory
47
48 1m 33s 2 {kubelet k8s-node2} spec.containers{nginx} Normal Pulled Container image "192.168.110.133:5000/nginx:1.13" already present on machine
49 33s 33s 1 {kubelet k8s-node2} spec.containers{nginx} Normal Killing Killing container with docker id ff079949b2cb: pod "exec_default(c69fbd14-b13e-11ea-80b4-000c2919d52d)" container "nginx" is unhealthy, it will be killed and re-created.
50 33s 33s 1 {kubelet k8s-node2} spec.containers{nginx} Normal Created Created container with docker id 54e08c980bd1; Security:[seccomp=unconfined]
51 33s 33s 1 {kubelet k8s-node2} spec.containers{nginx} Normal Started Started container with docker id 54e08c980bd1
52 [root@k8s-master health]# kubectl get all -o wide
53 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
54 rc/mysql 1 1 1 1h mysql 192.168.110.133:5000/mysql:5.7.30 app=mysql
55 rc/myweb 1 1 1 1h myweb 192.168.110.133:5000/tomcat:latest app=myweb
56
57 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
58 svc/kubernetes 10.254.0.1 <none> 443/TCP 13d <none>
59 svc/mysql 10.254.207.238 <none> 3306/TCP 1h app=mysql
60 svc/myweb 10.254.29.22 <nodes> 8080:30008/TCP 1h app=myweb
61
62 NAME READY STATUS RESTARTS AGE IP NODE
63 po/busybox2 1/1 Running 1 37m 172.16.16.3 k8s-master
64 po/exec 1/1 Running 1 2m 172.16.59.4 k8s-node2
65 po/mysql-lmx4s 1/1 Running 0 1h 172.16.59.2 k8s-node2
66 po/myweb-hsdwn 1/1 Running 0 1h 172.16.32.3 k8s-node3
67 [root@k8s-master health]# pod重启的次数，可以看到这个Pod的重启了多少次了。
4、创建httpGet的探针检测方法，liveness探针的httpGet使用。
1 [root@k8s-master health]# vim nginx_pod_httpGet.yaml具体内容，如下所示：
1 apiVersion: v1
2 kind: Pod
3 metadata:
4 name: httpget
5 spec:
6 containers:
7 - name: nginx
8 image: 192.168.110.133:5000/nginx:1.13
9 ports:
10 - containerPort: 80
11 livenessProbe:
12 httpGet:
13 path: /index.html
14 port: 80
15 initialDelaySeconds: 3
16 periodSeconds: 3
17 创建httpGet的探针检测方法，liveness探针的httpGet使用。
1 [root@k8s-master health]# kubectl get all -o wide
2 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
3 rc/mysql 1 1 1 1h mysql 192.168.110.133:5000/mysql:5.7.30 app=mysql
4 rc/myweb 1 1 1 1h myweb 192.168.110.133:5000/tomcat:latest app=myweb
5
6 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
7 svc/kubernetes 10.254.0.1 <none> 443/TCP 13d <none>
8 svc/mysql 10.254.207.238 <none> 3306/TCP 1h app=mysql
9 svc/myweb 10.254.29.22 <nodes> 8080:30008/TCP 1h app=myweb
10
11 NAME READY STATUS RESTARTS AGE IP NODE
12 po/busybox2 1/1 Running 1 46m 172.16.16.3 k8s-master
13 po/exec 0/1 CrashLoopBackOff 6 11m 172.16.59.4 k8s-node2
14 po/httpget 1/1 Running 0 22s 172.16.16.4 k8s-master
15 po/mysql-lmx4s 1/1 Running 0 1h 172.16.59.2 k8s-node2
16 po/myweb-hsdwn 1/1 Running 0 1h 172.16.32.3 k8s-node3
17 [root@k8s-master health]# kubectl describe pod httpget
18 Name: httpget
19 Namespace: default
20 Node: k8s-master/192.168.110.133
21 Start Time: Thu, 18 Jun 2020 16:47:03 +0800
22 Labels: <none>
23 Status: Running
24 IP: 172.16.16.4
25 Controllers: <none>
26 Containers:
27 nginx:
28 Container ID: docker://f9e0ea6a16a26d3fca57b1e1b521f642f89eb05e6951a05830933e68dce1253a
29 Image: 192.168.110.133:5000/nginx:1.13
30 Image ID: docker-pullable://docker.io/nginx@sha256:b1d09e9718890e6ebbbd2bc319ef1611559e30ce1b6f56b2e3b479d9da51dc35
31 Port: 80/TCP
32 State: Running
33 Started: Thu, 18 Jun 2020 16:47:04 +0800
34 Ready: True
35 Restart Count: 0
36 Liveness: http-get http://:80/index.html delay=3s timeout=1s period=3s #success=1 #failure=3
37 Volume Mounts: <none>
38 Environment Variables: <none>
39 Conditions:
40 Type Status
41 Initialized True
42 Ready True
43 PodScheduled True
44 No volumes.
45 QoS Class: BestEffort
46 Tolerations: <none>
47 Events:
48 FirstSeen LastSeen Count From SubObjectPath Type Reason Message
49 --------- -------- ----- ---- ------------- -------- ------ -------
50 1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned httpget to k8s-master
51 1m 1m 1 {kubelet k8s-master} spec.containers{nginx} Normal Pulled Container image "192.168.110.133:5000/nginx:1.13" already present on machine
52 1m 1m 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id f9e0ea6a16a2; Security:[seccomp=unconfined]
53 1m 1m 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id f9e0ea6a16a2
54 [root@k8s-master health]# 默认，这个首页index.html是可以访问到的，这里也可以让它访问不到的。
1 [root@k8s-master health]# kubectl exec -it httpget bash
2 root@httpget:/# cd /usr/share/html
3 bash: cd: /usr/share/html: No such file or directory
4 root@httpget:/# cd /usr/share/nginx/html
5 root@httpget:/usr/share/nginx/html# ls
6 50x.html index.html
7 root@httpget:/usr/share/nginx/html# mv index.html /tmp/
8 root@httpget:/usr/share/nginx/html# exit
9 exit
10 [root@k8s-master health]# kubectl get pod httpget
11 NAME READY STATUS RESTARTS AGE
12 httpget 1/1 Running 1 3m
13 [root@k8s-master health]# kubectl describe pod httpget
14 Name: httpget
15 Namespace: default
16 Node: k8s-master/192.168.110.133
17 Start Time: Thu, 18 Jun 2020 16:47:03 +0800
18 Labels: <none>
19 Status: Running
20 IP: 172.16.16.4
21 Controllers: <none>
22 Containers:
23 nginx:
24 Container ID: docker://7bb1ef7b33b24e16aad18053b3238077baac8e4b507ba85a6943dc9e9ae3ea29
25 Image: 192.168.110.133:5000/nginx:1.13
26 Image ID: docker-pullable://docker.io/nginx@sha256:b1d09e9718890e6ebbbd2bc319ef1611559e30ce1b6f56b2e3b479d9da51dc35
27 Port: 80/TCP
28 State: Running
29 Started: Thu, 18 Jun 2020 16:50:07 +0800
30 Last State: Terminated
31 Reason: Completed
32 Exit Code: 0
33 Started: Thu, 18 Jun 2020 16:47:04 +0800
34 Finished: Thu, 18 Jun 2020 16:50:06 +0800
35 Ready: True
36 Restart Count: 1
37 Liveness: http-get http://:80/index.html delay=3s timeout=1s period=3s #success=1 #failure=3
38 Volume Mounts: <none>
39 Environment Variables: <none>
40 Conditions:
41 Type Status
42 Initialized True
43 Ready True
44 PodScheduled True
45 No volumes.
46 QoS Class: BestEffort
47 Tolerations: <none>
48 Events:
49 FirstSeen LastSeen Count From SubObjectPath Type Reason Message
50 --------- -------- ----- ---- ------------- -------- ------ -------
51 3m 3m 1 {default-scheduler } Normal Scheduled Successfully assigned httpget to k8s-master
52 3m 3m 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id f9e0ea6a16a2; Security:[seccomp=unconfined]
53 3m 3m 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id f9e0ea6a16a2
54 3m 13s 2 {kubelet k8s-master} spec.containers{nginx} Normal Pulled Container image "192.168.110.133:5000/nginx:1.13" already present on machine
55 19s 13s 3 {kubelet k8s-master} spec.containers{nginx} Warning Unhealthy Liveness probe failed: HTTP probe failed with statuscode: 404
56 13s 13s 1 {kubelet k8s-master} spec.containers{nginx} Normal Killing Killing container with docker id f9e0ea6a16a2: pod "httpget_default(4857d09b-b140-11ea-80b4-000c2919d52d)" container "nginx" is unhealthy, it will be killed and re-created.
57 13s 13s 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id 7bb1ef7b33b2; Security:[seccomp=unconfined]
58 12s 12s 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id 7bb1ef7b33b2
59 [root@k8s-master health]# kubectl describe pod httpget
60 Name: httpget
61 Namespace: default
62 Node: k8s-master/192.168.110.133
63 Start Time: Thu, 18 Jun 2020 16:47:03 +0800
64 Labels: <none>
65 Status: Running
66 IP: 172.16.16.4
67 Controllers: <none>
68 Containers:
69 nginx:
70 Container ID: docker://7bb1ef7b33b24e16aad18053b3238077baac8e4b507ba85a6943dc9e9ae3ea29
71 Image: 192.168.110.133:5000/nginx:1.13
72 Image ID: docker-pullable://docker.io/nginx@sha256:b1d09e9718890e6ebbbd2bc319ef1611559e30ce1b6f56b2e3b479d9da51dc35
73 Port: 80/TCP
74 State: Running
75 Started: Thu, 18 Jun 2020 16:50:07 +0800
76 Last State: Terminated
77 Reason: Completed
78 Exit Code: 0
79 Started: Thu, 18 Jun 2020 16:47:04 +0800
80 Finished: Thu, 18 Jun 2020 16:50:06 +0800
81 Ready: True
82 Restart Count: 1
83 Liveness: http-get http://:80/index.html delay=3s timeout=1s period=3s #success=1 #failure=3
84 Volume Mounts: <none>
85 Environment Variables: <none>
86 Conditions:
87 Type Status
88 Initialized True
89 Ready True
90 PodScheduled True
91 No volumes.
92 QoS Class: BestEffort
93 Tolerations: <none>
94 Events:
95 FirstSeen LastSeen Count From SubObjectPath Type Reason Message
96 --------- -------- ----- ---- ------------- -------- ------ -------
97 3m 3m 1 {default-scheduler } Normal Scheduled Successfully assigned httpget to k8s-master
98 3m 3m 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id f9e0ea6a16a2; Security:[seccomp=unconfined]
99 3m 3m 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id f9e0ea6a16a2
100 3m 18s 2 {kubelet k8s-master} spec.containers{nginx} Normal Pulled Container image "192.168.110.133:5000/nginx:1.13" already present on machine
101 24s 18s 3 {kubelet k8s-master} spec.containers{nginx} Warning Unhealthy Liveness probe failed: HTTP probe failed with statuscode: 404
102 18s 18s 1 {kubelet k8s-master} spec.containers{nginx} Normal Killing Killing container with docker id f9e0ea6a16a2: pod "httpget_default(4857d09b-b140-11ea-80b4-000c2919d52d)" container "nginx" is unhealthy, it will be killed and re-created.
103 18s 18s 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id 7bb1ef7b33b2; Security:[seccomp=unconfined]
104 17s 17s 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id 7bb1ef7b33b2
105 [root@k8s-master health]# 5、liveness探针的tcpSocket使用。
1 [root@k8s-master health]# vim nginx_pod_tcpSocket.yaml使用tcpSocket监控的是80的端口，配置文件的内容如下所示：
1 apiVersion: v1
2 kind: Pod
3 metadata:
4 name: tcpsocket
5 spec:
6 containers:
7 - name: nginx
8 image: 192.168.110.133:5000/nginx:1.13
9 ports:
10 - containerPort: 80
11 livenessProbe:
12 tcpSocket:
13 port: 80
14 initialDelaySeconds: 3
15 periodSeconds: 3
16 创建，如下所示：
1 [root@k8s-master health]# kubectl create -f nginx_pod_tcpSocket.yaml
2 pod "tcpsocket" created
3 [root@k8s-master health]# kubectl get all -o wide
4 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
5 svc/kubernetes 10.254.0.1 <none> 443/TCP 13d <none>
6
7 NAME READY STATUS RESTARTS AGE IP NODE
8 po/busybox2 1/1 Running 1 54m 172.16.16.3 k8s-master
9 po/exec 0/1 CrashLoopBackOff 7 18m 172.16.59.4 k8s-node2
10 po/httpget 1/1 Running 1 8m 172.16.16.4 k8s-master
11 po/tcpsocket 1/1 Running 0 6s 172.16.59.2 k8s-node26、readiness探针的httpGet使用。
1 [root@k8s-master health]# vim nginx_rc_readiness.yaml具体内容，如下所示：
需要访问qiangge.html，如果不存在就一直加入不到svc中。
1 apiVersion: v1
2 kind: ReplicationController
3 metadata:
4 name: readiness
5 spec:
6 replicas: 2
7 selector:
8 app: readiness
9 template:
10 metadata:
11 labels:
12 app: readiness
13 spec:
14 containers:
15 - name: readiness
16 image: 192.168.110.133:5000/nginx:1.13
17 ports:
18 - containerPort: 80
19 readinessProbe:
20 httpGet:
21 path: /qiangge.html
22 port: 80
23 initialDelaySeconds: 3
24 periodSeconds: 3创建完rc之后，需要创建一个svc的，这里使用命令创建，不再使用配置文件进行创建了。
1 [root@k8s-master health]# kubectl expose rc readiness --port=80
2 service "readiness" exposed
3 [root@k8s-master health]# kubectl describe svc readiness
4 Name: readiness
5 Namespace: default
6 Labels: app=readiness
7 Selector: app=readiness
8 Type: ClusterIP
9 IP: 10.254.218.107
10 Port: <unset> 80/TCP
11 Endpoints:
12 Session Affinity: None
13 No events.
14 [root@k8s-master health]# kubectl get all -o wide
15 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
16 rc/readiness 2 2 0 2m readiness 192.168.110.133:5000/nginx:1.13 app=readiness
17
18 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
19 svc/kubernetes 10.254.0.1 <none> 443/TCP 13d <none>
20 svc/readiness 10.254.218.107 <none> 80/TCP 28s app=readiness
21
22 NAME READY STATUS RESTARTS AGE IP NODE
23 po/busybox2 1/1 Running 1 1h 172.16.16.3 k8s-master
24 po/exec 1/1 Running 11 28m 172.16.59.4 k8s-node2
25 po/httpget 1/1 Running 1 18m 172.16.16.4 k8s-master
26 po/readiness-66j6c 0/1 Running 0 2m 172.16.59.5 k8s-node2
27 po/readiness-l2v6z 0/1 Running 0 2m 172.16.32.3 k8s-node3
28 po/tcpsocket 1/1 Running 0 10m 172.16.59.2 k8s-node2
29 [root@k8s-master health]# 可以看到readiness已经启动了两个Pod了，但是后端节点里面是空的。此时可以创建一个指定的html文件，就可以了。
此时发现后端节点也不为空了，Pod也正常启动了，另外一个可以类似出来，就可以将两个Pod正常启动起来了。
1 [root@k8s-master health]# kubectl exec -it readiness-66j6c bash
2 root@readiness-66j6c:/# cd /usr/share/nginx/html/
3 root@readiness-66j6c:/usr/share/nginx/html# echo hello nginx > qiangge.html
4 root@readiness-66j6c:/usr/share/nginx/html# cat qiangge.html
5 hello nginx
6 root@readiness-66j6c:/usr/share/nginx/html# exit
7 exit
8 [root@k8s-master health]# kubectl describe svc readiness
9 Name: readiness
10 Namespace: default
11 Labels: app=readiness
12 Selector: app=readiness
13 Type: ClusterIP
14 IP: 10.254.218.107
15 Port: <unset> 80/TCP
16 Endpoints: 172.16.59.5:80
17 Session Affinity: None
18 No events.
19 [root@k8s-master health]# kubectl get all -o wide
20 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR
21 rc/readiness 2 2 1 6m readiness 192.168.110.133:5000/nginx:1.13 app=readiness
22
23 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
24 svc/kubernetes 10.254.0.1 <none> 443/TCP 13d <none>
25 svc/readiness 10.254.218.107 <none> 80/TCP 4m app=readiness
26
27 NAME READY STATUS RESTARTS AGE IP NODE
28 po/busybox2 1/1 Running 1 1h 172.16.16.3 k8s-master
29 po/exec 0/1 CrashLoopBackOff 11 32m 172.16.59.4 k8s-node2
30 po/httpget 1/1 Running 1 21m 172.16.16.4 k8s-master
31 po/readiness-66j6c 1/1 Running 0 6m 172.16.59.5 k8s-node2
32 po/readiness-l2v6z 0/1 Running 0 6m 172.16.32.3 k8s-node3
33 po/tcpsocket 1/1 Running 0 13m 172.16.59.2 k8s-node2
34 [root@k8s-master health]# 本文参与 腾讯云自媒体分享计划，分享自作者个人站点/博客。
原始发表：2020-06-21 ，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读