ELK-logstash-6.3.2部署

踏歌行

发布于 2020-10-15 11:39:56

3750

发布于 2020-10-15 11:39:56

　　Logstash 是一款强大的数据处理工具，它可以实现数据传输，格式处理，格式化输出，还有强大的插件功能，常用于日志处理。

1. logstash部署

1 [yun@mini04 software]$ pwd
2 /app/software
3 [yun@mini04 software]$ tar xf logstash-6.3.2.tar.gz 
4 [yun@mini04 software]$ mv logstash-6.3.2 /app/
5 [yun@mini04 software]$ cd /app/
6 [yun@mini04 ~]$ ln -s logstash-6.3.2/ logstash

2. 基本测试

[yun@mini04 bin]$ pwd
/app/logstash/bin
[yun@mini04 bin]$ ./logstash -e 'input { stdin{} } output { stdout{} }'  # 输入什么就打印什么 
Sending Logstash's logs to /app/logstash/logs which is now configured via log4j2.properties
[2018-08-21T20:42:43,017][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-08-21T20:42:43,912][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.3.2"}
[2018-08-21T20:42:46,861][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-08-21T20:42:47,143][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x6c40d7d2 run>"}
The stdin plugin is now waiting for input:
[2018-08-21T20:42:47,325][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-08-21T20:42:47,693][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
zhangsan    # 输入
{
          "host" => "mini04",
       "message" => "zhangsan",
    "@timestamp" => 2018-08-21T12:43:08.026Z,
      "@version" => "1"
}
lisi   # 输入
{
          "host" => "mini04",
       "message" => "lisi",
    "@timestamp" => 2018-08-21T12:43:11.412Z,
      "@version" => "1"
}
[yun@mini04 bin]$ ./logstash -e 'input{ stdin{} } output{ stdout{codec => rubydebug} }'  # 启用debug模式
…………
1111  # 输入
{
       "message" => "1111",
          "host" => "mini04",
    "@timestamp" => 2018-08-21T12:57:09.274Z,
      "@version" => "1"
}
OOOO  # 输入
{
       "message" => "OOOO",
          "host" => "mini04",
    "@timestamp" => 2018-08-21T12:57:55.289Z,
      "@version" => "1"
}

3. 使用配置文件

[yun@mini04 config]$ pwd
/app/logstash/config
[yun@mini04 config]$ cat demo.conf 
input{
  stdin{}
}

filter{
}

output{
  # es有3台，随便指定一台即可  也可以是多台如 ["127.0.0.1:9200","127.0.0.2:9200"]
  elasticsearch { 
    hosts => ["mini03:9200"] 
    index => "logstash-%{+YYYY.MM.dd}"
  }
 
  stdout { codec => rubydebug } 
}
[yun@mini04 config]$ /app/logstash/bin/logstash -f /app/logstash/config/demo.conf # 启动
………………
1111
{
          "host" => "mini04",
    "@timestamp" => 2018-08-21T13:40:05.051Z,
       "message" => "1111",
      "@version" => "1"
}
222
{
          "host" => "mini04",
    "@timestamp" => 2018-08-21T13:40:08.445Z,
       "message" => "222",
      "@version" => "1"
}
zhangsan
{
          "host" => "mini04",
    "@timestamp" => 2018-08-21T13:40:11.333Z,
       "message" => "zhangsan",
      "@version" => "1"
}
周八
{
          "host" => "mini04",
    "@timestamp" => 2018-08-21T13:40:15.523Z,
       "message" => "周八",
      "@version" => "1"
}