首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >ES缓存

ES缓存

作者头像
YG
发布2018-05-23 17:19:01
1.5K0
发布2018-05-23 17:19:01
举报
文章被收录于专栏:YG小书屋YG小书屋

node query cache

一个节点的所有shard共享一个缓存区。利用LRU算法替换缓存内容。

query cache缓存查询结果,但只缓存filter类型的查询。

可通过indices.queries.cache.size设置缓存的大小。

在5.1.1中移除了term query的缓存。因为term query和filter query二者查询时间相差不多。https://www.elastic.co/guide/en/elasticsearch/reference/5.1/release-notes-5.1.1.html 因此下面的查询是不会缓存的。

curl -XPOST 'localhost:9200/_search?pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
        "term" : { "user" : "Kimchy" } 
    }
}'

针对于filter执行的不需要计算排名的查询,官网的说明如下:https://www.elastic.co/guide/en/elasticsearch/guide/2.x/_finding_exact_values.html

Find matching docs. The term query looks up the term XHDK-A-1293-#fJ3 in the inverted index and retrieves the list of documents that contain that term. In this case, only document 1 has the term we are looking for. Build a bitset. The filter then builds a bitset--an array of 1s and 0s—that describes which documents contain the term. Matching documents receive a 1 bit. In our example, the bitset would be [1,0,0,0]. Internally, this is represented as a "roaring bitmap", which can efficiently encode both sparse and dense sets. Iterate over the bitset(s) Once the bitsets are generated for each query, Elasticsearch iterates over the bitsets to find the set of matching documents that satisfy all filtering criteria. The order of execution is decided heuristically, but generally the most sparse bitset is iterated on first (since it excludes the largest number of documents). Increment the usage counter. Elasticsearch can cache non-scoring queries for faster access, but its silly to cache something that is used only rarely. Non-scoring queries are already quite fast due to the inverted index, so we only want to cache queries we know will be used again in the future to prevent resource wastage. To do this, Elasticsearch tracks the history of query usage on a per-index basis. If a query is used more than a few times in the last 256 queries, it is cached in memory. And when the bitset is cached, caching is omitted on segments that have fewer than 10,000 documents (or less than 3% of the total index size). These small segments tend to disappear quickly anyway and it is a waste to associate a cache with them.

注意当segment的文档数量小于10000或者小于总index数量的3%时,查询是不会缓存的。 在博客http://www.jianshu.com/p/b5ff856f3190中有提到同样的请求前两次访问时间为38ms,但是第3和4次请求时,需要的时间为150ms。后面再请求时,时间为1ms。一直有疑问,上述的第四条介绍了原因,具体是因为只有对频繁访问的请求才会建bitmap,建bitmap的过程需要一定的时间。 会被filter的情况:

Frequently used filters will be cached automatically by Elasticsearch, to speed up performance. Filter context is in effect whenever a query clause is passed to a filter parameter, such as the filter or must_not parameters in the bool query, the filter parameter in the constant_score query, or the filter aggregation.

经过测试的有以下查询会被缓存: 1 range 2 bool/must_not 3 bool/filter 4 constant_score/filter 5 filter aggregation

shard request cache

shard-level类型的请求会缓存本地的结果到每个shard。 shard request cache会缓存以下内容:

By default, the requests cache will only cache the results of search requests where size=0, so it will not cache hits, but it will cache hits.total, aggregations, and suggestions.Most queries that use now (see Date Mathedit) cannot be cached.

即: 1 hits.total 2 aggregations 3 suggestions

针对于含有now的query,可通过https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-search-speed.html#_search_rounded_dates加速。

缓存的结果会随着shard的refresh而无效。因此越长的refresh interval,在不超出deadline的情况下缓存可用的时间就越长。当缓存满时,最近最少使用的缓存将被清除。

可以设置每个索引的请求是否cache:

curl -XPUT 'localhost:9200/my_index?pretty' -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index.requests.cache.enable": false
  }
}
'

也可以手动的设置每个请求是否缓存:

curl -XGET 'localhost:9200/my_index/_search?request_cache=true&pretty' -H 'Content-Type: application/json' -d'
{
  "size": 0,
  "aggs": {
    "popular_colors": {
      "terms": {
        "field": "colors"
      }
    }
  }
}
'

size大于0的请求将不会被缓存,即使手动的设置request_cache=true。

cache key

提交的json body被作为cache key。如果你的json body发生了改变,则不能利用缓存。即使是同一个请求,但是条件的顺序不同,也不行。 可通过indices.requests.cache.size: 2% 设置cache的大小。

本文参与 腾讯云自媒体分享计划,分享自作者个人站点/博客。
原始发表:2017.11.21 ,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体分享计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • node query cache
  • shard request cache
    • cache key
    相关产品与服务
    Elasticsearch Service
    腾讯云 Elasticsearch Service(ES)是云端全托管海量数据检索分析服务,拥有高性能自研内核,集成X-Pack。ES 支持通过自治索引、存算分离、集群巡检等特性轻松管理集群,也支持免运维、自动弹性、按需使用的 Serverless 模式。使用 ES 您可以高效构建信息检索、日志分析、运维监控等服务,它独特的向量检索还可助您构建基于语义、图像的AI深度应用。
    领券
    问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档