Before collecting logs of a Pod on a super node to CLS, you need to authorize a role to the service to ensure that logs can be uploaded to CLS normally:
Follow the steps below:
1. Log in to the Access Management Console > Roles.
2. Click Create Role on the "Role" page.
3. In "Select Role Entity", choose Tencent Cloud Services > Container Service (TKE) > Container Service - EKS Log Collection, and click Next. As shown in the image below:
4. Confirm role policy, and click Next.
5. Review role policy, and click Done to complete role configuration.
Configuring log collection
You need to enable TKE log collection feature and configure corresponding log collection rule when you finished service role authorization. For example, you need to specify workload collection and Pod labels collection. For more information, see Using CRD to Configure Log Collection via the Console.
Collect logs to Kafka
To collect logs from Pods on super nodes to self-built Kafka or CKafka, you can configure the corresponding log collection rules in the console or configure the CRD yourself, defining the collection source and consumer. Once the CRD configuration is complete, the built-in collector in the Pod will collect logs according to the rules.
The specific CRD configuration is as follows:
apiVersion: cls.cloud.tencent.com/v1
kind: LogConfig ## Default value
metadata:
name: test ## CRD resource name, unique within the cluster
spec:
kafkaDetail:
brokers: xxxxxx ## (Required) The broker address. Generally, it is domain name:port. If there are more than one address, separate them with ",".
topic: xxxxxx # Topic ID, which is required
messageKey:# Optional, specify the Pod field as the key to upload to the designated partition
valueFrom:
fieldRef:
fieldPath: metadata.name
timestampKey:## The key of timestamp. It defaults to @timestamp
timestampFormat:## The format of timestamp. It defaults to double.
inputDetail:
type: container_stdout ## Log collection type, including container_stdout (container standard output) and container_file (container file)
containerStdout:## Container standard output
namespace: default ## The Kubernetes namespace of the container to be collected. If not specified, it represents all namespaces.
allContainers:false## Whether to collect the standard output of all containers in the specified namespace
container: xxx ## Name of the container for log collection, can be left empty here
includeLabels:## Collect Pods with the specified labels
k8s-app: xxx ## Collect logs only from pods with the label "k8s-app=xxx". This field cannot be specified if workloads or allContainers=true are specified.
workloads:## Kubernetes workloads to which the Pods containing the containers to be collected belong
-namespace: prod ## Workload namespace
name: sample-app ## Workload name
kind: deployment ## Workload type. Supported values include deployment, daemonset, statefulset, job, and cronjob.
container: xxx ## Name of the container to collect. If left blank, it indicates all containers in the workload Pod will be collected.
containerFile:## File in the container
namespace: default ## The Kubernetes namespace of the container to be collected. A namespace must be specified.
container: xxx ## The name of the container of which the logs will be collected. The * can be used here.
includeLabels:## Collect Pods with the specified labels
k8s-app: xxx ## Collect logs only from pods with the label "k8s-app=xxx". This field cannot be specified if workload is specified.
workload:## The Kubernetes workload to which the container's Pod belongs for log collection
name: sample-app ## Workload name
kind: deployment ## Workload type. Supported values include deployment, daemonset, statefulset, job, and cronjob.
logPath: /opt/logs ## Log folder. Wildcards are not supported.
filePattern: app_.log ## Log file name. It supports the wildcards "" and "?". "*" matches multiple random characters, and "?" matches a single random character.
