作为集群管理员,查看集群中的某个pod log 是常规操作,让我们看看具体都有哪些方法吧。
本文以OpenShift 集群以及它的客户端`oc` 为例,使用K8s 集群和`kubectl`的同学也可以参考,使用方法基本一致。
第一种方式,直接使用oc 的子命令 logs, 比如查看marketplace-operator pod log,
$ oc get pods -n openshift-marketplace
NAME READY STATUS RESTARTS AGE
marketplace-operator-6f865d6bcf-j57n6 1/1 Running 2 (70m ago) 85m
$ oc logs marketplace-operator-6f865d6bcf-j57n6 -n openshift-marketplace
time="2024-03-01T06:18:17Z" level=info msg="Go Version: go1.19.13 X:strictfipsruntime"
time="2024-03-01T06:18:17Z" level=info msg="Go OS/Arch: linux/amd64"
...
可以通过 `oc logs --help` 来查看更多的有趣的用法,我经常用到的是 `--previous` ,用来debug pod 为什么会重启。
$ oc logs --help
Print the logs for a resource.
Supported resources are builds, build configs (bc), deployment configs (dc), and pods. When a pod is specified and has
more than one container, the container name should be specified via -c. When a build config or deployment config is
specified, you can view the logs for a particular version of it via --version.
If your pod is failing to start, you may need to use the --previous option to see the logs of the last attempt.
Examples:
# Start streaming the logs of the most recent build of the openldap build config
oc logs -f bc/openldap
# Start streaming the logs of the latest deployment of the mysql deployment config
oc logs -f dc/mysql
# Get the logs of the first deployment for the mysql deployment config. Note that logs
# from older deployments may not exist either because the deployment was successful
# or due to deployment pruning or manual deletion of the deployment
oc logs --version=1 dc/mysql
# Return a snapshot of ruby-container logs from pod backend
oc logs backend -c ruby-container
# Start streaming of ruby-container logs from pod backend
oc logs -f pod/backend -c ruby-container
Options:
--all-containers=false:
Get all containers' logs in the pod(s).
-c, --container='':
Print the logs of this container
-f, --follow=false:
Specify if the logs should be streamed.
--ignore-errors=false:
If watching / following pod logs, allow for any errors that occur to be non-fatal
--insecure-skip-tls-verify-backend=false:
Skip verifying the identity of the kubelet that logs are requested from. In theory, an attacker could provide
invalid log content back. You might want to use this if your kubelet serving certificates have expired.
--limit-bytes=0:
Maximum bytes of logs to return. Defaults to no limit.
--max-log-requests=5:
Specify maximum number of concurrent logs to follow when using by a selector. Defaults to 5.
--pod-running-timeout=20s:
The length of time (like 5s, 2m, or 3h, higher than zero) to wait until at least one pod is running
--prefix=false:
Prefix each log line with the log source (pod name and container name)
-p, --previous=false:
If true, print the logs for the previous instance of the container in a pod if it exists.
-l, --selector='':
Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2). Matching
objects must satisfy all of the specified label constraints.
--since=0s:
Only return logs newer than a relative duration like 5s, 2m, or 3h. Defaults to all logs. Only one of
since-time / since may be used.
--since-time='':
仅返回在指定日期 (RFC3339) 之后的日志。默认为所有日志。只能使用 since-time / since 之一。
--tail=-1:
Lines of recent log file to display. Defaults to -1 with no selector, showing all log lines otherwise 10, if a
selector is provided.
--timestamps=false:
Include timestamps on each line in the log output
--version=0:
View the logs of a particular build or deployment by version if greater than zero
第二种方式,登录到pod所在的node 上去查看 pod log file。
$ oc get pods marketplace-operator-6f865d6bcf-j57n6 -o wide -n openshift-marketplace
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
marketplace-operator-6f865d6bcf-j57n6 1/1 Running 2 (107m ago) 122m 10.128.0.14 xxx-2mkn4-master-1 <none> <none>
$ oc debug node/xxx-2mkn4-master-1
Temporary namespace openshift-debug-bcr2j is created for debugging node...
Starting pod/weinliu-1310-2mkn4-master-1-debug-7mcd9 ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.0.5
If you don't see a command prompt, try pressing enter.
sh-4.4# chroot /host
sh-5.1#
sh-5.1# cat /var/log/pods/openshift-marketplace_marketplace-operator-6f865d6bcf-j57n6_a471f288-df08-4c3e-9ddb-072daf300a56/marketplace-operator/
1.log 2.log
sh-5.1# cat /var/log/pods/openshift-marketplace_marketplace-operator-6f865d6bcf-j57n6_a471f288-df08-4c3e-9ddb-072daf300a56/marketplace-operator/1.log
2024-03-01T06:12:48.400484292+00:00 stderr F time="2024-03-01T06:12:48Z" level=info msg="Go Version: go1.19.13 X:strictfipsruntime"
2024-03-01T06:12:48.400484292+00:00 stderr F time="2024-03-01T06:12:48Z" level=info msg="Go OS/Arch: linux/amd64"
...
第三种方式,`oc adm must-gather` 它会收集集群中所有的pod log,这对debug cluster issue 很有帮助,但是单独查看某个pod log 有点杀鸡用牛刀的感觉
$ oc adm must-gather --help
Launch a pod to gather debugging information.
This command will launch a pod in a temporary namespace on your cluster that gathers debugging information and then
downloads the gathered information.
Examples:
# Gather information using the default plug-in image and command, writing into ./must-gather.local.<rand>
oc adm must-gather
# Gather information with a specific local folder to copy to
oc adm must-gather --dest-dir=/local/directory
# Gather audit information
oc adm must-gather -- /usr/bin/gather_audit_logs
# Gather information using multiple plug-in images
oc adm must-gather --image=quay.io/kubevirt/must-gather --image=quay.io/openshift/origin-must-gather
# Gather information using a specific image stream plug-in
oc adm must-gather --image-stream=openshift/must-gather:latest
# Gather information using a specific image, command, and pod directory
oc adm must-gather --image=my/image:tag --source-dir=/pod/directory -- myspecial-command.sh
Options:
--dest-dir='':
Set a specific directory on the local machine to write gathered data to.
--host-network=false:
Run must-gather pods as hostNetwork: true - relevant if a specific command and image needs to capture
host-level data
--image=[]:
Specify a must-gather plugin image to run. If not specified, OpenShift's default must-gather image will be
used.
--image-stream=[]:
Specify an image stream (namespace/name:tag) containing a must-gather plugin image to run.
--node-name='':
Set a specific node to use - by default a random master will be used
--node-selector='':
Set a specific node selector to use - only relevant when specifying a command and image which needs to capture
data on a set of cluster nodes simultaneously
--run-namespace='':
An existing privileged namespace where must-gather pods should run. If not specified a temporary namespace
will be generated.
--source-dir='/must-gather/':
Set the specific directory on the pod copy the gathered data from.
--timeout='10m':
The length of time to gather data, like 5s, 2m, or 3h, higher than zero. Defaults to 10 minutes.
--volume-percentage=30:
Specify maximum percentage of must-gather pod's allocated volume that can be used. If this limit is exceeded,
must-gather will stop gathering, but still copy gathered data. Defaults to 30%!
大家平时都是如何查看pod log的呢?遇到过什么趣事吗?欢迎评论区留言!