Data Lake Compute (DLC) offers monitoring services for data engines based on the Tencent Cloud Observability Platform, ensuring real-time understanding of the operation of your data engines and the ability to configure data alarms. For alarm configuration methods, see Monitoring Alarm Configuration.
Notes
Before using the monitoring service of Data Lake Compute (DLC), you need to activate the Tencent Cloud Observability Platform service (for details on how to use the Tencent Cloud Observability Platform, see Tencent Cloud Observability Platform Documentation). If you have not yet activated this service, you can use the master account to do so.
Relevant charges may apply during the use of the Tencent Cloud Observability Platform service. For detailed billing information, see Billing Overview.
Monitoring Access Point
Access Point One: Data Lake Compute Console
Note:
The account must have monitoring permissions for the data engine.
2. From the left menu, select Cloud Product Monitoring, find Data Lake Compute (DLC), and choose the type of monitoring you wish to view.
3. After selecting the type of monitoring, you will be directed to the monitoring page. Select the corresponding region to view the monitoring resource information in that region.
4. Click on the Engine ID to access the monitoring details.
Configuration of Monitoring Granularity
You can configure the time range, granularity, and auto-update time range for monitoring data at the top of the monitoring page.
Monitoring Data Time Range: Accurate to the minute, it supports the selection of data over a specific period.
Time Granularity: The interval between monitoring points, which can be configured to either 1 minute or 5 minutes.
Automatic Data Update: The page data auto-refresh configuration supports settings for disabling, 30s, 5min, 30min, and 1h intervals.
Monitoring Data Comparison
It supports selecting a period of time for monitoring comparison. After clicking to select the comparison time range, you can view the comparison data in the data compass below.
Monitored metrics
Test mode
Monitored metrics
CPU
Maximum CPU Utilization of All Driver Nodes
Maximum CPU Utilization of All Executor Nodes
Average CPU Utilization of All Driver Nodes
Average CPU Utilization of All Executor Nodes
Maximum CPU Utilization of All Clusters
Average CPU Utilization of All Clusters
Memory
Maximum Memory Utilization of All Driver Nodes
Maximum Memory Utilization Across All Executor Nodes
Average Memory Utilization Across All Driver Nodes
Average Memory Utilization Across All Executor Nodes
Max Memory Utilization of All Clusters
Average Memory Usage Across All Clusters
Task scheduling
Number of Cancelled Tasks
Number of Failed Tasks
Number of Initialization Tasks
Average Initialization Duration of Tasks
Maximum Task Initialization Duration
Number of Queued Tasks
Average Queue Time for Tasks
Maximum Task Queue Duration
Number of Running Tasks
Number of Successful Tasks
Networking
Maximum Network Inbound Bandwidth for All Driver Nodes
Maximum Network Inbound Bandwidth of All Executor Nodes
Average inbound network bandwidth of all Driver nodes
Average Inbound Bandwidth of All Executor Nodes
Maximum Network Outbound Bandwidth of All Driver Nodes
Maximum Network Outbound Bandwidth of All Executor Nodes
Average Outbound Bandwidth of All Driver Nodes
Average Outbound Bandwidth of All Executor Nodes
Cloud Disk
Maximum Utilization of All Driver Node Cloud Disks
Maximum Usage Rate of Cloud Disk for All Executor Nodes
Average Usage Rate of Cloud Disks for All Driver Nodes
Average Cloud Disk Utilization of All Executor Nodes
