我正在使用logstash将日志提供给ElasticSearch。我将logstash输出配置为:
input {
file {
path => "/tmp/foo.log"
codec =>
plain {
format => "%{message}"
}
}
}
output {
elasticsearch {
#host => localhost
codec => json {}
manage_template => false
index => "4glogs"
}
}
我注意到,只要我启动logstash,它就会在ES中创建一个映射(日志),如下所示。
{
"4glogs": {
"mappings": {
"logs": {
"properties": {
"@timestamp": {
"type": "date",
"format": "dateOptionalTime"
},
"@version": {
"type": "string"
},
"message": {
"type": "string"
}
}
}
}
}
}
如何防止logstash创建此映射?
更新:
我现在也解决了这个错误。“日志的对象映射试图解析为对象,但获得EOF,是否为其提供了具体的值?”
正如John Petrone在下面所述,一旦定义了映射,就必须确保您的文档符合该映射。在我的例子中,我定义了一个"type: nested“的映射,但是logstash的输出是一个字符串。因此,我从logstash配置中删除了所有编解码器(无论是json还是plain ),这使得json文档无需更改即可通过。
这是我的新logstash配置(带有一些额外的多行日志过滤器)。
input {
kafka {
zk_connect => "localhost:2181"
group_id => "logstash_group"
topic_id => "platform-logger"
reset_beginning => false
consumer_threads => 1
queue_size => 2000
consumer_id => "logstash-1"
fetch_message_max_bytes => 1048576
}
file {
path => "/tmp/foo.log"
}
}
filter {
multiline {
pattern => "^\s"
what => "previous"
}
multiline {
pattern => "[0-9]+$"
what => "previous"
}
multiline {
pattern => "^$"
what => "previous"
}
mutate{
remove_field => ["kafka"]
remove_field => ["@version"]
remove_field => ["@timestamp"]
remove_tag => ["multiline"]
}
}
output {
elasticsearch {
manage_template => false
index => "4glogs"
}
}
https://stackoverflow.com/questions/24924248
复制相似问题