StarRocks Monitoring Metrics

Last updated: 2023-12-21 16:53:24

StarRocks-BE

Title
Description
Metric Unit
Description
Compaction Deltas Data Volume
Cumulative
rowsets
Data Volume of Cumulative Compaction Deltas
Base
rowsets
Data Volume of Base Compaction Deltas
Update
rowsets
Data Volume of Update Compaction Deltas
Volume of COMPACTION Data
Cumulative
bytes
Data Volume of Cumulative Compaction Bytes
Base
bytes
Data Volume of Base Compaction Bytes
Update
bytes
Data Volume of Update Compaction Bytes
Maximum Score of TABLET COMPACTION
CumulativeMax
score
Maximum Base Compaction Score within Tablet
BaseMax
score
Maximum Compaction Score of Tablet Base
ENGINE Request Failure Statistics (1)
base_compaction
count
Number of Engine Failure Requests of Type Base_Compaction
clone
count
Number of Engine Failure Requests of Type Clone
create_rollup
count
Number of Engine Failure Requests of Type Create_Rollup
create_tablet
count
Number of Engine Failure Requests of Type Create_Tablet
cumulative_compaction
count
Number of Engine Failure Requests of Type Cumulative_Compaction
delete
count
Number of Engine Failure Requests of Type Delete
finish_task
count
Number of Engine Failure Requests of Type Finish_Task
Engine Request Failure Statistics (2)
publish
count
Number of Engine Failure Requests of Type Publish
report_all_tablets
count
Number of Engine Failure Requests of Type Report_All_Tablets
report_disk
count
Number of Engine Failure Requests of Type Report_Disk
report_tablet
count
Number of Engine Failure Requests of Type Report_Tablet
report_task
count
Number of Engine Failure Requests of Type Report_Task
schema_change
count
Number of Engine Failure Requests of Type Schema_Change
Engine Request Statistics (1)
base_compaction
count
Number of Engine Failure Requests of Type Base_Compaction
clone
count
Number of Engine Failure Requests of Type Clone
create_rollup
count
Number of Engine Failure Requests of Type Create_Rollup
create_tablet
count
Number of Engine Failure Requests of Type Create_Tablet
cumulative_compaction
count
Number of Engine Failure Requests of Type Cumulative_Compaction
delete
count
Number of Engine Failure Requests of Type Delete
drop_tablet
count
Number of Engine Failure Requests of Type Drop_Tablet
finish_task
count
Number of Engine Failure Requests of Type Finish_Task
Engine Request Statistics (2)
publish
count
Number of Engine Failure Requests of Type Publish
report_all_tablets
count
Number of Engine Failure Requests of Type Report_All_Tablets
report_disk
count
Number of Engine Failure Requests of Type Report_Disk
report_tablet
count
Number of Engine Failure Requests of Type Report_Tablet
report_task
count
Number of Engine Failure Requests of Type Report_Task
schema_change
count
Number of Engine Failure Requests of Type Schema_Change
storage_migrate
count
Number of Engine Failure Requests of Type Storage_Migrate
FRAGMENT Statistics
PlanFragment
count
Plan Fragment Quantity
Endpoint
count
Quantity of DataStreams
FRAGMENT Request Time
Duration
Microsecond (μs)
Request Time of Fragment
TXN Request Statistics
begin
count
Quantity of Requests with 'begin' as TXN Type
commit
count
Quantity of Requests with 'commit' as TXN Type
exec
count
Quantity of Requests with 'exec' as TXN Type
rollback
count
Quantity of Requests with 'rollback' as TXN Type
STREAMING LOAD Data Volume
LoadTotal
bytes
Data Size Imported via Stream Load
STREAMING LOAD Statistics
CurrentProcessing
count
Current Process Count for Streaming Load
PipeCount
count
Number of Pipes in Streaming Load
STREAMING LOAD Duration
Duration
ms
Streaming Load Duration
BE Memory
Total
bytes
Size of BE Memory Pool
Allocated
bytes
Size of Allocated BE Memory
Number of Process File Handles
Used
count
Number of File Handles Utilized by BE Process
SoftLimit
count
Soft Limit Quantity of File Handles for BE Process
HardLimit
count
Hard Limit Quantity of File Handles for BE Process
Number of Running Threads in Process
Thread
count
Number of Threads Running in the BE Process
Quantity of THRIFT Utilization
Broker
count
Number of Brokers Utilizing Thrift
Backend
count
Number of BEs Utilizing Thrift
Frontend
count
Number of FEs Utilizing Thrift
TABLET Write Statistics
Writer
count
BE TABLET Write Statistics

StarRocks-FE

Title
Description
Metric Unit
Description
ALTER Task Statistics
RollupRunning
count
Number of Alter Jobs in Progress, of Type ROLLUP
SchemaChangeRunning
count
Number of Alter Jobs in Progress, of Type SCHEMA_CHANGE
IMAGE Statistics
Write
count
FE Quantity of Image Writes
Push
count
FE Quantity of Image Pushes
Quantity of SCHEDULED_TABLET
ScheduledTablet
count
Number of Scheduled Tablets in FE
TRANSACTION Status Statistics
Reject
count
FE Number of Rejected Transactions
Begin
count
FE Number of Initiated Transactions
Success
count
FE Number of Successful Transactions
Failed
count
FE Number of Failed Transactions
JVM Memory_HEAP
max
bytes
Maximum Heap Memory
committed
bytes
Committed Heap Memory
used
bytes
Utilized Heap Memory
JVM Memory_NONHEAP
committed
bytes
Committed Non-Heap Memory
used
bytes
Utilized Non-Heap Memory
JVM Memory_OLD
used
bytes
Utilized Old Generation Memory
peak_used
bytes
Maximum Utilized Old Generation Memory
max
bytes
Maximum Old Generation Memory
JVM Memory_YOUNG
used
bytes
Utilized Young Memory
peak_used
bytes
Maximum Utilized Young Memory
max
bytes
Maximum Young Memory
ROUTINE LOAD QUEUE Dimension
report queue
count
Size of the FE Report Queue
Number of Rows in ROUTINE_LOAD
TotalRows
count
Number of Rows in FE Routine Load
ErrorRows
count
FE Routine Load Number of Erroneous Rows
Size of ROUTINE LOAD
Receive
bytes
Size of the FE Routine Load
Peak Score of TABLET_COMPACTION
MAX
score
Maximum Compaction Score during FE Tablet Compaction
EDITLOG Write Delay
Quantile75
ms
75th Percentile of FE Editlog Write Delay
Quantile95
ms
95th Percentile of FE Editlog Write Delay
Quantile98
ms
98th Percentile of FE Editlog Write Delay
Quantile99
ms
99th Percentile of FE Editlog Write Delay
Quantile999
ms
99.9th Percentile of FE Editlog Write Delay
Garbage Collection Frequency
YoungGC
count
Number of Young GC Events in JVM of FE Nodes
OldGC
count
FE Node JVM Old GC Event Count
Garbage Collection Duration
YoungGC
Second.
Duration of Young GC Events in JVM of FE Nodes
OldGC
Second.
Duration of Old GC Events in JVM of FE Nodes
Number of JVM Threads
Total
count
Total Number of Threads in FE Node's JVM
Peak
count
Peak JVM Threads of FE Node
BROKER_LOAD Task Statistics
UNKNOWN
count
Quantity of Load Jobs of BROKER Type, in UNKNOWN Status
PENDING
count
Quantity of Load Jobs of BROKER Type, in PENDING Status
ETL
count
Quantity of Load Jobs of BROKER Type, in ETL Status
LOADING
count
Quantity of Load Jobs of BROKER Type, in LOADING Status
COMMITTED
count
Quantity of Load Jobs of BROKER Type, in COMMITTED Status
FINISHED
count
Quantity of Load Jobs of BROKER Type, in FINISHED Status
CANCELLED
count
Quantity of Load Jobs of BROKER Type, in CANCELLED Status
DELETE_LOAD Task Statistics
UNKNOWN
count
Quantity of Load Jobs of DELETE Type, in UNKNOWN Status
PENDING
count
Quantity of Load Jobs of DELETE Type, in PENDING Status
ETL
count
Quantity of Load Jobs of DELETE Type, in ETL Status
LOADING
count
Quantity of Load Jobs of DELETE Type, in LOADING Status
COMMITTED
count
Quantity of Load Jobs of DELETE Type, in COMMITTED Status
FINISHED
count
Quantity of Load Jobs of DELETE Type, in FINISHED Status
CANCELLED
count
Quantity of Load Jobs of DELETE Type, in CANCELLED Status
HADOOP_LOAD Task Statistics
UNKNOWN
count
Quantity of Load Jobs of HADOOP Type, in UNKNOWN Status
PENDING
count
Quantity of Load Jobs of HADOOP Type, in PENDING Status
ETL
count
Quantity of Load Jobs of HADOOP Type, in ETL Status
LOADING
count
Quantity of Load Jobs of HADOOP Type, in LOADING Status
COMMITTED
count
Quantity of Load Jobs of HADOOP Type, in COMMITTED Status
FINISHED
count
Quantity of Load Jobs of HADOOP Type, in FINISHED Status
CANCELLED
count
Quantity of Load Jobs of HADOOP Type, in CANCELLED Status
INSERT_LOAD Task Statistics
UNKNOWN
count
Quantity of Load Jobs of INSERT Type, in UNKNOWN Status
PENDING
count
Quantity of Load Jobs of INSERT Type, in PENDING Status
ETL
count
Quantity of Load Jobs of INSERT Type, in ETL Status
LOADING
count
Quantity of Load Jobs of INSERT Type, in LOADING Status
COMMITTED
count
Quantity of Load Jobs of INSERT Type, in COMMITTED Status
FINISHED
count
Quantity of Load Jobs of INSERT Type, in FINISHED Status
CANCELLED
count
Quantity of Load Jobs of INSERT Type, in CANCELLED Status
ROUTINE_LOAD Task Statistics
NEED_SCHEDULE
count
Routine Load Jobs Statistics, State=NEED_SCHEDULE
RUNNING
count
Routine Load Jobs Statistics, State=RUNNING
PAUSED
count
Routine Load Jobs Statistics, State=PAUSED
STOPPED
count
Routine Load Jobs Statistics, State=STOPPED
CANCELLED
count
Routine Load Jobs Statistics, State=CANCELLED
SPARK_LOAD Task Statistics
UNKNOWN
count
Quantity of Load Jobs of SPARK Type, in UNKNOWN Status
PENDING
count
Quantity of Load Jobs of SPARK Type, in PENDING Status
ETL
count
Quantity of Load Jobs of SPARK Type, in ETL Status
LOADING
count
Quantity of Load Jobs of SPARK Type, in LOADING Status
COMMITTED
count
Quantity of Load Jobs of SPARK Type, in COMMITTED Status
FINISHED
count
Quantity of Load Jobs of SPARK Type, in FINISHED Status
CANCELLED
count
Quantity of Load Jobs of SPARK Type, in CANCELLED Status
FE MASTER
FE Master
count
Indication of FE Master Status; 1 for Master, 0 for Follower
Node Information
FeNodeNum
count
Total Number of FE Nodes
BeNodeNum
count
Total Number of BE Nodes
BeAliveNum
count
Number of Active BE Nodes
BeDecommissionedNum
count
Number of Active BE Nodes
BkDeadNum
count
Number of Deceased Broker Nodes
Request Response
QPS
count/s
Queries Per Second
RPS
count/s
Number of Requests Processable Per Second
FE Query Statistics
total
count
Total Number of FE Queries
err
count
Total Number of FE Query Errors
timeout
count
Number of FE Query Timeouts
success
count
Total Number of Successful FE Queries
slow
count
Total Number of Slow FE Queries
Query Failure Rate
ErrRate
%
Query Error Rate
Query Latency in FE Nodes
Quantile75
ms
75th Percentile of FE Query Latency
Quantile95
ms
95th Percentile of FE Query Latency
Quantile99
ms
99th Percentile of FE Query Latency
Quantile999
ms
99.9th Percentile of FE Query Latency
Number of CONNECTIONS
Num
count
FE Node Connection Quantity

StarRocks-Broker

Title
Description
Metric Unit
Description
CPU utilization
ProcessCpuLoad
%
Process CPU Utilization
CPU Usage Duration
ProcessCpuTime
ms
Accumulated CPU Usage Time
Garbage Collection Frequency
YGC
Threads
Young Garbage Collection Frequency
FGC
Threads
Full Garbage Collection Frequency
Garbage Collection Duration
GCT
s
Garbage Collection Time Expenditure
FGCT
s
Full Garbage Collection Time Consumption
YGCT
s
Young Garbage Collection Time Consumption
Memory Region Occupancy
O
%
Old Region Memory Usage Proportion
M
%
Metaspace Region Memory Usage Proportion
CCS
%
Compressed Class Space Region Memory Usage Proportion
S0
%
Survivor 0 Region Memory Usage Proportion
S1
%
Survivor 1 Region Memory Usage Proportion
E
%
Eden Region Memory Usage Proportion
JVM Memory
MemHeapInitM
MB
Initial Quantity of JVM Heap Memory
MemNonHeapInitM
MB
Initial Quantity of JVM NonHeapMemory
MemHeapMaxM
MB
Configured Quantity of HeapMemory in JVM
MemHeapCommittedM
MB
Quantity of Currently Committed HeapMemory in JVM
MemHeapUsedM
MB
Current Quantity of HeapMemory in Use by JVM
MemNonHeapCommittedM
MB
Quantity of Currently Committed NonHeapMemory in JVM
MemNonHeapUsedM
MB
Current Quantity of NonHeapMemory in Use by JVM
Number of File Handles
OpenFileDescriptorCount
count
Number of Open File Descriptors
MaxFileDescriptorCount
count
Maximum File Descriptor Quantity
Process Runtime Duration
Uptime
s
Process Runtime Duration
Number of Working Threads
PeakThreadCount
count
Peak Thread Count
ThreadCount
count
Total Thread Count
DaemonThreadCount
count
Daemon Thread Quantity