python elasticsearch 批量写入测试

sparkexpert

发布于 2019-05-26 14:09:05

1.7K0

发布于 2019-05-26 14:09:05

文章被收录于专栏：大数据智能实战大数据智能实战

elasticsearch 是一个高效的近实时的数据库。最近要重新进行老版本的ES 数据迁移到新版本。由于以前都是采用java来进行连接的，不过这回为了方便，找了个python 连接包来进行处理。

本博客主要测试python　elasticsearch　进行批量写入的测试

步骤一：安装python的elasticsearch包

用：pip install elasticsearch

步骤二：批量写入测试：

import datetime# import datetime as
from datetime import timezone
import time
from random import randint

from elasticsearch import helpers
from elasticsearch import Elasticsearch

def WriteES():
    es = Elasticsearch()
    
    base = datetime.datetime.today()
    numdays = 100
    
    j = 0
    actions = []
    while (j <= 100):
        d1 = base - datetime.timedelta(days = j)
        ts= int(time.mktime(d1.timetuple())*1000)
        action = {
            "_index": "tickets",
            "_type": "last",
            "_id": j,
            "_source": {
                "count":randint(0,1000),
                "timestamp": ts
                }
            }
        actions.append(action)
        j += 1
    
    helpers.bulk(es, actions)

if __name__ == '__main__':
    WriteES()
    print('work done!')

该步骤基本上属于秒级完成。

步骤三：写入结果查看