The content of this page has been automatically translated by AI. If you encounter any problems while reading, you can view the corresponding content in Chinese.
Reasonable index settings are the foundation for efficient and stable business operations. In reality, index management not only has a high usage threshold and requires significant maintenance investment, but is also the root cause of many online issues. Currently, 60% of maintenance management operations and 60% of basic online issues with ES are related to this, making it a critical pain point in the use of ES.
Against this background, Tencent Cloud ES has introduced the industry-leading autonomous indexing, which can fully automate index creation, rolling, tiering, deletion, and automatic fault repair. It also achieves intelligent shard optimization, significantly reducing maintenance failures and resource investment.
Optimized Scheme
It provides features like automatic shard optimization, query pruning, automatic fault repair, and index lifecycle management. At the same time, it optimizes the writing layer to reduce maintenance and management costs, while improving usage efficiency and performance in writing and querying.
Search for the required CAM policy as needed, and click to complete policy association.
Optimization Effect
Overall, the autonomous indexing encompasses the entire index usage process, from index creation > data write > shard tuning > inquiry > operations management. It provides corresponding features to enhance the usage and management of indexes.
Efficient Creation and Management
The creation and management of autonomous indexes are more efficient compared to native indexes and Datastream. Autonomous indexes have built-in index lifecycle management policies, index templates, etc. All configurations can be completed with a single command, without the additional cost of managing ILM, index templates, or aliases.
Write performance
In response to issues such as "jitter in writes due to 0-index scrolling", "millions of TPS with CPU underutilization", and "more than double the performance degradation in writes with inherent primary keys", Autonomous Index has made corresponding optimizations at the write layer. Compared to the open-source version of ES, these optimizations have resulted in an 80% improvement in write stability, more than double the increase in TPS, a 50+% boost in resource utilization, and a 45% increase in performance. The data is as follows:
Comparison Metric
Tencent Cloud ES
Open Source Version
Write Throughput (TPS)
169W(+121%)
76W
CPU
49%(+58%)
31%
Rejection Rate
0%
0.3%
Automatic Shard Tuning
Autonomous Indexing supports automatic shard adjustments based on business load, compatible with rapid write growth, periodic fluctuations, write spikes, etc. It is linked with the node count, ensuring that in scenarios like rapid write scaling and scaling, there will be no write rejections or the need for manual adjustments.
Query Cropping
Autonomous indexing can use query conditions to quickly skip irrelevant indexes, reduce the number of shard requests, achieve index-level fast cropping, and enhance PB-level log query performance by more than 3 times.
Search for the required CAM policy as needed, and click to complete policy association.
Automatic Failure Repair
When a failure is detected in the node where an index shard is located, causing the index to turn red or write abnormalities, autonomous indexing will automatically roll out new backup indexes. These new backup index shards are distributed on normal nodes, ensuring the availability of write operations. The entire process requires no manual intervention and is seamless to business operations, all handled automatically by autonomous indexing.
Search for the required CAM policy as needed, and click to complete policy association.
