ES索引管理工具 - curator
原创
官方文档：https://www.elastic.co/guide/en/elasticsearch/client/curator/current/actions.html
社区好文：https://blog.csdn.net/laoyang360/article/details/85882832
curator与es版本的兼容性：https://www.elastic.co/guide/en/elasticsearch/client/curator/current/version-compatibility.html
curator允许对索引和快照执行许多不同的操作，包括：
1. 从别名添加或删除索引（或两者！）
2. 更改分片路由分配更改分片路由分配
3. 关闭索引关闭索引
4. 创建索引创建索引
5. 删除索引删除索引
6. 删除快照删除快照
7. 打开被关闭的索引打开被关闭的索引
8. 对索引执行forcemerge段合并操作对索引执行forcemerge段合并操作
9. reindex索引，包括来自远程集群的索引reindex索引，包括来自远程集群的索引
10. 更改索引的每个分片的副本数 更改索引的每个分片的副本数
11. rollover索引rollover索引
12. 生成索引的快照（备份）生成索引的快照（备份）
13. 还原快照还原快照使用方法
安装
pip install elasticsearch-curator命令行下单次执行
curator_cli --host 127.0.0.1 --port 9200 close --filter_list '[{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":1},{"filtertype":"pattern","kind":"prefix","value":"logs_"}]'更常用的是，定时任务执行：
mkdir /root/.curator
cd /root/.curatorcat curator.yml 内容如下：
client:
hosts: 192.168.2.187
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile: /var/log/curator.log
logformat: default
blacklist: ['elasticsearch', 'urllib3']cat action.yml 内容如下：
# 下面我这里演示的是关闭7天前的索引
actions:
1:
action: close
description: "Close selected indices"
options:
ignore_empty_list: True
disable_action: false
skip_flush: false
delete_aliases: false
filters:
- filtertype: pattern
kind: prefix
value: 'logs_|filebeat-|logstash-services-|metricsbeat-'
- filtertype: age
source: name
direction: older
timestring: '%Y-%m-%d'
unit: days
unit_count: 7由于支持的action操作非常多，建议直接参考官网配置即可：
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/actions.html
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/examples.html执行：
格式：curator [--config CONFIG.YML] [--dry-run] ACTION_FILE.YML
curator --config curator.yml action.yml添加定时任务：
50 23 * * * cd /root/.curator/ && curator --config curator.yml action.yml
tail -F /var/log/curator.log 日志类似如下：2019-11-04 23:14:13,089 INFO Action ID: 1, "close" completed.
2019-11-04 23:14:13,089 INFO Job completed.
2019-11-04 23:14:30,934 INFO Preparing Action ID: 1, "close"
2019-11-04 23:14:30,935 INFO Creating client object and testing connection
2019-11-04 23:14:30,941 INFO Instantiating client object
2019-11-04 23:14:30,942 INFO Testing client connectivity
2019-11-04 23:14:30,949 INFO Successfully created Elasticsearch client object with provided settings
2019-11-04 23:14:30,953 INFO Trying Action ID: 1, "close": Close selected indices
2019-11-04 23:14:30,995 INFO Closing 1 selected indices: ['metricsbeat-2019-11-01']
2019-11-04 23:14:31,164 INFO Action ID: 1, "close" completed.
2019-11-04 23:14:31,165 INFO Job completed.
2019-11-04 23:18:06,032 INFO Instantiating client object
2019-11-04 23:18:06,033 INFO Testing client connectivity
2019-11-04 23:18:06,041 INFO Successfully created Elasticsearch client object with provided settings一个组合的写法(也可以将多个action拆分成多个脚本来执行)：
actions:
1:
action: create_index
description: "Create index as named"
options:
continue_if_exception: True
name: '<logstash-{now/d+1d}>'
extra_settings:
settings:
number_of_shards: 2
number_of_replicas: 0
2:
action: close
description: "Close selected indices"
options:
ignore_empty_list: True
disable_action: false
skip_flush: false
delete_aliases: false
filters:
- filtertype: pattern
kind: prefix
value: 'logs_|filebeat-|logstash-services-|metricsbeat-'
- filtertype: age
source: name
direction: older
timestring: '%Y-%m-%d'
unit: days
unit_count: 7
3:
action: delete_indices
description: "Delete selected indices"
options:
continue_if_exception: False
timeout_override: 300
filters:
- filtertype: pattern
kind: prefix
value: 'logs_|filebeat-|logstash-services-|metricsbeat-'
- filtertype: age
source: name
direction: older
timestring: '%Y-%m-%d'
unit: days
unit_count: 14脚本正常执行时候，日志如下：
2019-11-05 00:42:06,377 INFO Preparing Action ID: 1, "create_index"
2019-11-05 00:42:06,377 INFO Creating client object and testing connection
2019-11-05 00:42:06,384 INFO Instantiating client object
2019-11-05 00:42:06,385 INFO Testing client connectivity
2019-11-05 00:42:06,392 INFO Successfully created Elasticsearch client object with provided settings
2019-11-05 00:42:06,396 INFO Trying Action ID: 1, "create_index": Create index as named
2019-11-05 00:42:06,396 INFO "<logstash-{now/d+2d}>" is using Elasticsearch date math.
2019-11-05 00:42:06,396 INFO Creating index "<logstash-{now/d+2d}>" with settings: {'settings': {'number_of_shards': 2, 'number_of_replicas': 0}}
2019-11-05 00:42:06,608 INFO Action ID: 1, "create_index" completed.
2019-11-05 00:42:06,608 INFO Preparing Action ID: 2, "close"
2019-11-05 00:42:06,608 INFO Creating client object and testing connection
2019-11-05 00:42:06,609 INFO Instantiating client object
2019-11-05 00:42:06,609 INFO Testing client connectivity
2019-11-05 00:42:06,615 INFO Successfully created Elasticsearch client object with provided settings
2019-11-05 00:42:06,619 INFO Trying Action ID: 2, "close": Close selected indices
2019-11-05 00:42:06,672 INFO Closing 3 selected indices: ['filebeat-2019-10-02', 'filebeat-2018-10-25', 'filebeat-2019-10-25']
2019-11-05 00:42:06,875 INFO Action ID: 2, "close" completed.
2019-11-05 00:42:06,875 INFO Preparing Action ID: 3, "delete_indices"
2019-11-05 00:42:06,875 INFO Creating client object and testing connection
2019-11-05 00:42:06,876 INFO Instantiating client object
2019-11-05 00:42:06,876 INFO Testing client connectivity
2019-11-05 00:42:06,881 INFO Successfully created Elasticsearch client object with provided settings
2019-11-05 00:42:06,885 INFO Trying Action ID: 3, "delete_indices": Delete selected indices
2019-11-05 00:42:06,927 INFO Deleting 2 selected indices: ['filebeat-2018-10-25', 'filebeat-2019-10-02']
2019-11-05 00:42:06,927 INFO ---deleting index filebeat-2018-10-25
2019-11-05 00:42:06,927 INFO ---deleting index filebeat-2019-10-02
2019-11-05 00:42:07,007 INFO Action ID: 3, "delete_indices" completed.
2019-11-05 00:42:07,007 INFO Job completed.切记：
curator适用于基于时间或者template其他方式创建的索引，不适合单一索引存储N久历史数据的操作的场景。
原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。
如有侵权，请联系 cloudcommunity@tencent.com 删除。
原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。
如有侵权，请联系 cloudcommunity@tencent.com 删除。
评论
登录后参与评论
推荐阅读
相关产品与服务
Elasticsearch Service
腾讯云 Elasticsearch Service（ES）是云端全托管海量数据检索分析服务，拥有高性能自研内核，集成X-Pack。ES 支持通过自治索引、存算分离、集群巡检等特性轻松管理集群，也支持免运维、自动弹性、按需使用的 Serverless 模式。使用 ES 您可以高效构建信息检索、日志分析、运维监控等服务，它独特的向量检索还可助您构建基于语义、图像的AI深度应用。