集群节点与pods
non-running
的pods(k是kubectl的命令别名)☸️ CloudnativeEcosystem? default ~ ? ? k get pods -A --field-selector=status.phase!=Running | \
grep -v Complete
☸️ CloudnativeEcosystem? default ~ ? ? k get no -o json | \
jq -r '.items | sort_by(.status.capacity.memory)[]|[.metadata.name,.status.capacity.memory]| @tsv'
master.devopsman.cn 2029760Ki
node1.devopsman.cn 2029760Ki
node2.devopsman.cn 2029760Ki
☸️ CloudnativeEcosystem? default ~ ? ? k get po -o json --all-namespaces | \
jq '.items | group_by(.spec.nodeName) | map({"nodeName": .[0].spec.nodeName, "count": length}) | sort_by(.count)'
[
{
"nodeName": "node1.devopsman.cn",
"count": 4
},
{
"nodeName": "master.devopsman.cn",
"count": 8
},
{
"nodeName": "node2.devopsman.cn",
"count": 11
}
]
Kubectl top
获取消耗CPU和内存资源的Pod列表的方法,可以按需排序☸️ CloudnativeEcosystem? default ~ ? ? k top pods -A | sort --reverse --key 3 --numeric
...
kube-system coredns-66bff467f8-7hktk 2m 18Mi
default traefik-577bb96ffb-tx2p2 2m 32Mi
kube-system metrics-server-58f6f5594b-rsh8h 1m 13Mi
kube-system kube-sealyun-lvscare-node2.devopsman.cn 1m 10Mi
kube-system kube-proxy-msrxn 1m 27Mi
kube-system kube-proxy-fts6g 1m 23Mi
kube-system calico-kube-controllers-84445dd79f-zbtk4 1m 14Mi
default nginx-744f4df6df-8wmmj 0m 6Mi
NAMESPACE NAME CPU(cores) MEMORY(bytes)
restart
的次数进行排序,当然还可以使用其他状态进行排序☸️ CloudnativeEcosystem? default ~ ? ? bash -c "kubectl get pods --sort-by=.status.containerStatuses[0].restartCount"
NAME READY STATUS RESTARTS AGE
nginx-744f4df6df-8wmmj 1/1 Running 1 14d
traefik-577bb96ffb-tx2p2 1/1 Running 1 14d
dev-cluster-nfs-client-provisioner-755dcfbb-4fp86 1/1 Running 5 14d
6. 获取所有pods的limits
和requests
,并自定义列表头
☸️ CloudnativeEcosystem? default ~ ? ? k get pods -A -o=custom-columns='NAME:spec.containers[*].name,MEMREQ:spec.containers[*].resources.requests.memory,MEMLIM:spec.containers[*].resources.limits.memory,CPUREQ:spec.containers[*].resources.requests.cpu,CPULIM:spec.containers[*].resources.limits.cpu'
NAME MEMREQ MEMLIM CPUREQ CPULIM
...
dev-cluster-nfs-client-provisioner <none> <none> <none> <none>
nginx 128Mi 128Mi 500m 500m
coredns 70Mi 170Mi 100m <none>
coredns 70Mi 170Mi 100m <none>
coredns 70Mi 170Mi 100m <none>
etcd <none> <none> <none> <none>
explain
会帮助你获取最正确的信息☸️ CloudnativeEcosystem? default ~ ? ? k explain sts KIND: StatefulSet
VERSION: apps/v1 # apiVersion
DESCRIPTION:
StatefulSet represents a set of pods with consistent identities. Identities
are defined as:
- Network: A single stable DNS and hostname.
- Storage: As many VolumeClaims as requested. The StatefulSet guarantees
that a given network identity will always map to the same storage identity.
internal IP
的方法☸️ CloudnativeEcosystem? default ~ ? ? k get nodes -o json | \
jq -r '.items[].status.addresses[]? | select (.type == "InternalIP") | .address' | \
paste -sd "\n" -
192.168.99.128
192.168.99.133
192.168.99.134
svc
以及各自的nodePort
☸️ CloudnativeEcosystem? default ~ ? ? k get --all-namespaces svc -o json | \
jq -r '.items[] | [.metadata.name,([.spec.ports[].nodePort | tostring ] | join("|"))]| @tsv'
kubernetes null
nginx-svc null
traefik null|null|null
traefikudp 30667
kube-dns null|null|null
metrics-server null
rmq-cluster null
rmq-cluster-balancer 31489|32051
flannel
或者其他的容器网络插件CNI
出现问题的时候,检查pod的路由信息是很重要的,下面列出pod的子网☸️ CloudnativeEcosystem? default ~ ? ? k get nodes -o jsonpath='{.items[*].spec.podCIDR}' | tr " " "\n"
100.64.0.0/24
100.64.2.0/24
100.64.1.0/24
☸️ CloudnativeEcosystem? default ~ ? ? k logs -f nginx-744f4df6df-8wmmj --timestamps
2020-10-05T08:43:32.3816926Z /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
2020-10-05T08:43:32.381727847Z /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
2020-10-05T08:43:32.387449329Z /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
☸️ CloudnativeEcosystem? default ~ ? ? k logs -f nginx-744f4df6df-8wmmj --timestamps --tail=50
2020-10-05T08:43:32.3816926Z /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
2020-10-05T08:43:32.381727847Z /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
2020-10-05T08:43:32.387449329Z /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
2020-10-05T08:43:32.398286535Z 10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
2020-10-05T08:43:32.419095055Z 10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
2020-10-05T08:43:32.419117067Z /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
2020-10-05T08:43:32.42342319Z /docker-entrypoint.sh: Configuration complete; ready for start up
container
的日志☸️ CloudnativeEcosystem? default ~ ? ? k logs -f nginx-744f4df6df-8wmmj --timestamps --tail=50 --all-containers
2020-10-05T08:43:32.3816926Z /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
2020-10-05T08:43:32.381727847Z /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
2020-10-05T08:43:32.387449329Z /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
2020-10-05T08:43:32.398286535Z 10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
previous
容器的日志,比如已经crashed
的☸️ CloudnativeEcosystem? default ~ ? ? k logs --help |grep 'previous'
# Return snapshot of previous terminated ruby container logs from pod web-1
-p, --previous=false: If true, print the logs for the previous instance of the container in a pod if it exists.
☸️ CloudnativeEcosystem? default ~ ? ? k logs -f nginx-744f4df6df-8wmmj --timestamps --tail=50 --previous
2020-10-05T05:16:40.322078192Z /docker-entrypoint.sh: Configuration complete; ready for start up
2020-10-05T05:17:31.117726669Z 100.110.232.4 - - [05/Oct/2020:05:17:31 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like G
2020-10-05T05:17:31.433713897Z 100.110.232.4 - - [05/Oct/2020:05:17:31 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://web.devopsman.cn/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36" "192.168.99.1"
2020-10-05T05:17:49.607617864Z 100.110.232.4 - - [05/Oct/2020:05:17:49 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36" "192.168.99.1"
☸️ CloudnativeEcosystem? default ~ ? ? k logs -f -l app=nginx
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh