Rally 最初于 2016 年发布,直到 2018 年 7 月才发布 1.0 版本,Rally 工具 是 Elasticsearch 开发团队用来运行夜间基准测试的工具。
Rally 的美妙之处在于它不仅可以充当负载生成器,还可以为构建、设置Elasticsearch 集群,从而帮助我们进行性能测试。
我们可以使用 Rally 对现有的 Elasticsearch 集群进行基准测试、管理基准配置、运行和比较结果,并使用指标和报错(例如 JIT、GC、perf)发现潜在的性能问题。
Rally 并发默认是配置多进程,基础进程数 8+1(client数量)。一台服务器只能运行一个 Rally 实例。
Rally 的 git 地址:
https://github.com/elastic/rally
下文中的:esrally 等价于 Rally 工具。
track:是赛道的意思,在这里是指压测用的数据和测试策略,track.json 便是压测策略的定义文件。
operations: 指定具体的操作,比如 index 索引数据的操作、force-merge 强制合并 segment 的操作、search 搜索的操作等等。
schedule: 指定了 challenge
中按顺序执行的 task,其中每个 task 都指定了 一个 operation
,除此之外还可以设定 clients (并发客户端数)、warmup-iterations(预热的循环次数)、iterations(operation 执行的循环次数)等。
指定特定的 schedule
可以使用命令行参数--include-tasks
和exclude-tasks
challenges: 通过组合 schedule
定义一系列 task ,再组合成一个压测的流程。
指定 challenges
可以使用命令行参数--challenge
pipeline:指的是压测的流程。在生产中,一般都是远端集群,因此主要选择 benchmark-only。
在 Metric 一栏,是压测结果指标数据,一般要关注的数据有:
注意:
target-interval
或target-throughput
的一个,但不能两者都定义(否则 Rally 将引发错误)。iterations
与time-period
这两类的参数也不能同时定义。基础配置
{
"schedule": [
{
"operation": {
"operation-type": "search",
"index": "_all",
"body": {
"query": {
"match_all": {}
}
}
},
"warmup-iterations": 100,
"iterations": 100,
"target-throughput": 10
}
]
}
并发配置 默认 rally 对每个 operation 分配一个 client 进行处理。
在 parallel 和每个 task 都可以指定 clients。
如果两者的clients参数出现冲突,则可参考以下示例:
"schedule": [
{
"parallel": {
"warmup-iterations": 50,
"iterations": 100,
"clients": 2,
"tasks": [
{
"operation": "match-all",
"target-throughput": 50
},
{
"operation": "term",
"target-throughput": 200
},
{
"operation": "phrase",
"target-throughput": 200
}
]
}
}
]
Rally 不会同时运行所有三个任务,Rally 将首先同时运行“match-all”和“term”(每个任务一个client)。在他们完成之后,Rally将与一个客户运行“phrase”。
如果 parallel 指定比任务更多的 client,但这些 client 将是空闲的。
{
"parallel": {
"clients": 3,
"warmup-iterations": 50,
"iterations": 100,
"tasks": [
{
"operation": "default",
"target-throughput": 50
},
{
"operation": "term",
"target-throughput": 200
},
{
"operation": "phrase",
"target-throughput": 200,
"clients": 2
}
]
}
}
这种方式确保 phrase 由两个 client 执行,其他任务一个client执行。
yum -y install curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker libffi-devel bzip2-devel
wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.7.5.tar.gz --no-check-certificate
wget https://www.python.org/ftp/python/3.8.7/Python-3.8.7.tgz
cd Python-3.8.7/
./configure prefix=/usr/local/python3
make && make install
echo 'export PYTHON3_HOME=/usr/local/python3' >> /etc/profile
echo 'export PATH=$PATH:$PYTHON3_HOME/bin' >> /etc/profile
source /etc/profile
pip3.8 install sklearn
tar -zxvf git-2.7.5.tar.gz
cd git-2.7.5/
make prefix=/usr/local/esrally/git all
make prefix=/usr/local/esrally/git install
rpm -qa | grep -w git
rpm -e git-1.8.3.1-23.el7_8.x86_64 --nodeps
echo 'export GIT2_HOME=/usr/local/esrally/git' >> /etc/profile
echo 'export PATH=$PATH:$GIT2_HOME/bin' >> /etc/profile
source /etc/profile
wget https://github.com/elastic/rally/releases/esrally-dist-linux-2.3.1.tar.gz --no-check-certificate
tar -zxf esrally-dist-linux-2.3.0.tar.gz
cd esrally-dist-2.3.0/
bash install.sh
如果是已存在python3其他版本,后安装的python3.8,需要修改 install.sh中的python3修改成python3.8
默认rally参数和压测文件路径:/root/.rally/
使用 /root/.rally/benchmarks/tracks/default/download.sh 选择下载离线测试数据
安装后测试
esrally race --track=metricbeat --challenge=append-no-conflicts --target-hosts=127.0.0.1:9200 --client-options="use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'passwd'" --pipeline=benchmark-only --offline
这里分三个场景:
其中查询的dsl为:
{"query":{"term":{"meta.cloud.instance_id":{"value":"1983702708814995873"}}}}
创建track路径
/usr/local/esrally/tracks/acms
获取索引的mapping文件
#命令格式
curl -k --user 用户名:密码 -XGET http://<es域名>:9200/<索引名>/_mapping?pretty=true > <索引名>.json
#实际命令
curl -k --user elastic:abc123 -XGET http://127.0.0.1:9200/metricbeat/_mapping?pretty=true > metricbeat.json
创建track.json
{% import "rally.helpers" as rally with context %}
{
"version": 2,
"description": "Tracker-generated track for test",
"indices": [
{
"name": "metricbeat", --------索引名
"body": "metricbeat.json" --------索引mapping文件名
}
],
"schedule": [
{
"operation": {
"name": "query-dsl",
"operation-type": "search",
"body": {
"query": {
"term": {
"meta.cloud.instance_id": {
"value": "1983702708814995873"
}
}
}
}
},
"warmup-time-period": 10, ------warmup10s
"time-period": 120, ------实际测试120s
"target-throughput": 4500,------目标吞吐量
"clients": 5 ------启动的客户端数量
}
]
}
运行自定义查询测试
esrally race --track-path=/usr/local/esrally/tracks/acme --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --client-options="use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'abc123'" --include-tasks="query-dsl" --report-file=/usr/local/esrally/report/20220128-5-10k.csv
使用 --report-file 定义测试结果输出文件。
创建 track ,获取压测数据,此处会生产索引数据文件,因此需要注意track路径的文件使用。
esrally create-track --track=acme --target-hosts=127.0.0.1:9200 --client-options="use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'passwd'" --indices="metricbeat" --output-path=/usr/local/esrally/tracks
在 track.json 中添加 operation
{% import "rally.helpers" as rally with context %}
{
"version": 2,
"description": "Tracker-generated track for acme",
"indices": [
{
"name": "metricbeat",
"body": "metricbeat.json"
}
],
"corpora": [
{
"name": "metricbeat",
"documents": [
{
"target-index": "metricbeat",
"source-file": "metricbeat-documents.json.bz2",
"document-count": 1079600,
"compressed-bytes": 97535962,
"uncompressed-bytes": 1249705758
}
]
}
],
"schedule": [
{
"operation": "delete-index"
},
{
"operation": {
"operation-type": "create-index",
"settings": {{index_settings | default({}) | tojson}}
}
},
{
"operation": {
"operation-type": "cluster-health",
"index": "metricbeat",
"request-params": {
"wait_for_status": "{{cluster_health | default('green')}}",
"wait_for_no_relocating_shards": "true"
},
"retry-until-success": true
}
},
{
"operation": {
"operation-type": "bulk",
"bulk-size": {{bulk_size | default(5000)}},
"ingest-percentage": {{ingest_percentage | default(100)}}
},
"clients": {{bulk_indexing_clients | default(8)}}
},
#### 此处新增
{
"operation": {
"name": "query-dsl",
"operation-type": "search",
"body": {
"query": {
"term": {
"meta.cloud.instance_id": {
"value": "1983702708814995873"
}
}
}
}
},
"warmup-time-period": 60,
"time-period": 120,
"target-throughput": 450,
"clients": 5
}
]
}
修改索引 metricbeat 的分片数,即修改 metricbeat.json 文件中分片参数
"number_of_shards": "12"
重建索引
esrally race --track-path=/usr/local/esrally/tracks/acme --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --client-options="use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'passwd'" --exclude-tasks="type:search"
测试自定义查询
esrally race --track-path=/usr/local/esrally/tracks/acme --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --client-options="use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'passwd'" --include-tasks="query-dsl" --report-file=/usr/local/esrally/report/20220128-5-10k.csv
创建track的方式与3.2一致。 以添加 challenges 文件的方式增加schedule;
#####原来track.json
{% import "rally.helpers" as rally with context %}
{
"version": 2,
"description": "Tracker-generated track for acme",
"indices": [
{
"name": "metricbeat",
"body": "metricbeat.json"
}
],
"corpora": [
{
"name": "metricbeat",
"documents": [
{
"target-index": "metricbeat",
"source-file": "metricbeat-documents.json.bz2",
"document-count": 1079600,
"compressed-bytes": 97377230,
"uncompressed-bytes": 1249705758
}
]
}
],
"schedule": [
{
"operation": "delete-index"
},
{
"operation": {
"operation-type": "create-index",
"settings": {{index_settings | default({}) | tojson}}
}
},
{
"operation": {
"operation-type": "cluster-health",
"index": "metricbeat",
"request-params": {
"wait_for_status": "{{cluster_health | default('green')}}",
"wait_for_no_relocating_shards": "true"
},
"retry-until-success": true
}
},
{
"operation": {
"operation-type": "bulk",
"bulk-size": {{bulk_size | default(5000)}},
"ingest-percentage": {{ingest_percentage | default(100)}}
},
"clients": {{bulk_indexing_clients | default(8)}}
}
]
}
#####修改后
{% import "rally.helpers" as rally with context %}
{
"version": 2,
"description": "Tracker-generated track for acme",
"indices": [
{
"name": "metricbeat",
"body": "metricbeat.json"
}
],
"corpora": [
{
"name": "metricbeat",
"documents": [
{
"target-index": "metricbeat",
"source-file": "metricbeat-documents.json.bz2",
"document-count": 1079600,
"compressed-bytes": 97377230,
"uncompressed-bytes": 1249705758
}
]
}
],
"challenges": [ #####此处添加
{{ rally.collect(parts="challenges/*.json") }}
]
}
在/usr/local/esrally/tracks/acme路径下添加 challenges 路径,在其中新建query-dsl.json。
{
"name": "query-dsl",
"default": false,
"schedule": [{
"operation": {
"name": "query-dsl",
"operation-type": "search",
"body": {
"query": {
"term": {
"meta.cloud.instance_id": {
"value": "1983702708814995873"
}
}
}
}
},
"warmup-time-period": 60,
"time-period": 120,
"target-throughput": 450,
"clients": 5
}]
}
将原来的 operations 迁移到default.json,设置 "default": true 使其成为默认的 challenges。
{
"name": "default",
"default": true,
"schedule": [
{
"operation": "delete-index"
},
{
"operation": {
"operation-type": "create-index",
"settings": {{index_settings | default({}) | tojson}}
}
},
{
"operation": {
"operation-type": "cluster-health",
"index": "metricbeat",
"request-params": {
"wait_for_status": "{{cluster_health | default('green')}}",
"wait_for_no_relocating_shards": "true"
},
"retry-until-success": true
}
},
{
"operation": {
"operation-type": "bulk",
"bulk-size": {{bulk_size | default(5000)}},
"ingest-percentage": {{ingest_percentage | default(100)}}
},
"clients": {{bulk_indexing_clients | default(8)}}
}
]
}
修改索引 metricbeat 的分片数,即修改 metricbeat.json 文件中分片参数
"number_of_shards": "12"
重建索引
esrally race --track-path=/usr/local/esrally/tracks/acme --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --client-options="use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'passwd'"
测试自定义查询:
esrally race --track-path=/usr/local/esrally/tracks/acme --challenge=query-dsl --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --client-options="use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'passwd'" --report-file=/usr/local/esrally/report/20220128-5-10k.csv
使用 --report-file 定义测试结果输出文件。
track
进行测试内容定义的时候,如果仅仅是对查询内容的qps测试,则不需要create-track
这个命令项创建track
,直接创建track.json
文件来减少测试的时间和存储成本。bulk
和search
的流程,建议定义不同的challenges
进行区分。esrally
单 client 的 bulk 写入性能不到2w docs/s
,如果需增加压力则需要加 client。latency
减去 service_time
即为 rally 等待 Elasticsearch 的响应时间。作者:金多安,Elastic 认证工程师、Elastic 中文社区日报编辑、运维工程师。
审稿人:铭毅天下,Elastic 认证工程师,Elastic 中国合作培训讲师,阿里云 MVP,Elastic Stack 技术博文全网累计阅读量 1000万+。
本文分享自 铭毅天下Elasticsearch 微信公众号,前往查看
如有侵权,请联系 cloudcommunity@tencent.com 删除。
本文参与 腾讯云自媒体同步曝光计划 ,欢迎热爱写作的你一起参与!