作者:郑增权 原创内容未经授权不得随意使用,转载请联系小编并注明来源。
本文基于 Pod 、Service 和 Ingress 三大模块进行划分,对于 Kubernetes 日常可能出现的故障问题,提供了较为具体的排查步骤,并附上相关解决方法或参考文献。
[root@10-186-65-37 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-deploy-55b54d55b8-5msx8 0/1 Pending 0 5m
[root@10-186-65-37 ~]# kubectl describe quota compute-resources --namespace=myspace
Name: compute-resources
Namespace: myspace
Resource Used Hard
-------- ---- ----
limits.cpu 0 2
limits.memory 0 2Gi
pods 0 4
requests.cpu 0 1
requests.memory 0 1Gi
[root@10-186-65-37 k8s-file]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
local-device-pvc Pending local-device 72s
则参考如下链接进行解决:https://kubernetes.io/zh/docs/concepts/storage/persistent-volumes/
[root@10-186-65-37 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
myapp-deploy-55b54d55b8-5msx8 1/1 Running 0 14d 10.244.4.9 10-186-65-122 <none> <none>
myapp-deploy-55b54d55b8-7ldj4 1/1 Running 0 14d 10.244.2.10 10-186-65-126 <none> <none>
myapp-deploy-55b54d55b8-cwdwt 1/1 Running 0 14d 10.244.3.9 10-186-65-126 <none> <none>
myapp-deploy-55b54d55b8-gvmb9 1/1 Running 0 14d 10.244.4.10 10-186-65-122 <none> <none>
myapp-deploy-55b54d55b8-xbqb6 1/1 Running 0 14d 10.244.5.9 10-186-65-118 <none> <none>
则是 Scheduler 方面的问题,参考如下链接进行解决:https://kubernetes.io/zh/docs/concepts/scheduling-eviction/kube-scheduler/
若能正确获取日志则根据日志修复相关问题。
[root@10-186-65-37 ~]# kubectl logs myapp-deploy-55b54d55b8-5msx8
127.0.0.1 - - [30/Sep/2021:06:53:16 +0000] "GET / HTTP/1.1" 200 65 "-" "curl/7.29.0" "-"
127.0.0.1 - - [30/Sep/2021:07:49:44 +0000] "GET / HTTP/1.1" 200 65 "-" "curl/7.29.0" "-"
127.0.0.1 - - [30/Sep/2021:07:51:09 +0000] "GET / HTTP/1.1" 200 65 "-" "curl/7.29.0" "-"
127.0.0.1 - - [30/Sep/2021:07:57:00 +0000] "GET / HTTP/1.1" 200 65 "-" "curl/7.29.0" "-"
127.0.0.1 - - [30/Sep/2021:08:03:56 +0000] "GET / HTTP/1.1" 200 65 "-" "curl/7.29.0" "-"
[root@10-186-65-37 ~]# docker history fb4cca6b4e4c
IMAGE CREATED CREATED BY SIZE COMMENT
fb4cca6b4e4c 22 months ago /bin/sh -c #(nop) COPY file:957630e64c05c549… 121MB
<missing> 2 years ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
<missing> 2 years ago /bin/sh -c #(nop) ADD file:1d711f09b1bbc7c8d… 42.3MB
[root@10-186-65-37 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-deploy-55b54d55b8-5msx8 1/1 Running 0 14d
myapp-deploy-55b54d55b8-7ldj4 1/1 Running 0 14d
若无处于 READY 状态的 pods 则前往 2.1.11 。
2. kubectl port-forward <pod-name> 8080:<pod-port>
3. 映射成功前往 2.2 。
a)进行映射
[root@10-186-65-37 ~]# kubectl port-forward myapp-deploy-55b54d55b8-5msx8 8080:80
Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80
b)验证映射成功
[root@10-186-65-37 ~]# curl localhost:8080
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
kubectl port-forward --address 0.0.0.0 <pod-name> 8080:<pod-port>
若无法被所有地址监听则为未知状态(Unknown state)。
[root@10-186-65-37 ~]# kubectl describe service myapp
\Name: myapp
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"myapp","namespace":"default"},"spec":{"ports":[{"name":"http","po...
Selector: app=myapp,release=canary
Type: ClusterIP
IP: 10.96.109.76
Port: http 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.2.10:80,10.244.3.9:80,10.244.4.10:80 + 2 more...
Session Affinity: None
Events: <none>
[root@10-186-65-37 ~]# kubectl port-forward service/myapp 8080:80
Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80
[root@10-186-65-37 ~]# kubectl describe pod myapp-deploy-55b54d55b8-5msx8 | grep -i label -A 2
Labels: app=myapp
pod-template-hash=55b54d55b8
release=canary
[root@10-186-65-37 ~]# kubectl describe service myapp | grep -i selector
Selector: app=myapp,release=canary
[root@10-186-65-37 ~]# kubectl describe pod myapp-deploy-55b54d55b8-5msx8 | grep -i 'ip'
IP: 10.244.4.9
IPs:
IP: 10.244.4.9
[root@10-186-65-37 ~]# kubectl describe service myapp | grep -i targetport
TargetPort: 80/TCP
[root@10-186-65-37 ~]# kubectl describe pod myapp-deploy-55b54d55b8-5msx8 | grep -i port
Port: 80/TCP
Host Port: 0/TCP
[root@10-186-65-37 ~]# kubectl describe ingress ingress-tomcat-tls
Name: ingress-tomcat-tls
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
TLS:
tomcat-ingress-secret terminates tomcat.quan.com
Rules:
Host Path Backends
---- ---- --------
tomcat.quan.com
tomcat:8080 (10.244.2.11:8080,10.244.4.11:8080,10.244.5.10:8080)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx"},"name":"ingress-tomcat-tls","namespace":"default"},"spec":{"rules":[{"host":"tomcat.quan.com","http":{"paths":[{"backend":{"serviceName":"tomcat","servicePort":8080},"path":null}]}}],"tls":[{"hosts":["tomcat.quan.com"],"secretName":"tomcat-ingress-secret"}]}}
kubernets.io/ingress.class: nginx
Events: <none>
[root@10-186-65-37 ~]# kubectl describe ingress ingress-tomcat-tls | grep -E 'serviceName|servicePort'
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx"},"name":"ingress-tomcat-tls","namespace":"default"},"spec":{"rules":[{"host":"tomcat.quan.com","http":{"paths":[{"backend":{"serviceName":"tomcat","servicePort":8080},"path":null}]}}],"tls":[{"hosts":["tomcat.quan.com"],"secretName":"tomcat-ingress-secret"}]}}
可正常访问前往 2.3.5 ,否则前往 2.3.3 。