logstash配置codec插件-JSON模式

配置nginx日志
log_format json '{"remote_addr":"$remote_addr" ,"host":"$host" ,"server_addr":"$server_addr" ,"timestamp":"$time_iso8601" ,"request_time":$request_time, "remote_user":"$remote_user",  "request":"$request" ,"status":$status, "body_sent":$body_bytes_sent ,"http_referer":"$http_referer" ,"http_user_agent":"$http_user_agent" ,"http_x_forwarded_for":"$http_x_forwarded_for"}';
配置logstash
input {
	file {
		path => ["/data/logs/nginx/collectd.dev-access.log"]
		type => "demo-codec-json-log"
		start_position => "beginning"
        codec => "json"
	}
}
output {
	stdout{
		codec=>rubydebug
	}
}
启动
bin/logstash -f /etc/logstash/conf.d/demo-codec-json.conf
结果
{
             "remote_addr" => "192.168.56.1",
                 "request" => "GET /graph.php?p=load&t=load&h=192.168.56.201&s=86400 HTTP/1.1",
                    "type" => "demo-codec-json-log",
             "server_addr" => "192.168.56.201",
         "http_user_agent" => "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36",
             "remote_user" => "-",
                    "path" => "/data/logs/nginx/collectd.dev-access.log",
            "request_time" => 0.026,
              "@timestamp" => 2017-06-13T06:31:12.761Z,
            "http_referer" => "http://collectd.dev/host.php?h=192.168.56.201&p=load",
                    "host" => "collectd.dev",
    "http_x_forwarded_for" => "-",
                "@version" => "1",
               "body_sent" => 13863,
               "timestamp" => "2017-06-13T06:31:12+00:00",
                  "status" => 200
}
备注
nginx日志当中部分字段可能会是数字或者-，可以将日志全部转换为字符串，然后通过filter来处理