自定义 Filebeat 数据接入

最近更新时间:2024-09-24 17:58:01

我的收藏

自建 Filebeat 数据采集

版本说明 仅支持7.10.2或者7.14.2的 Filebeat 版本。
类别
参数项
参数描述
填写说明
Elasticsearch template setting
setup.template.enabled
索引模板
布尔类型,设置为 false,目前不支持设置
setup.ilm.enabled
索引生命周期管理
布尔类型,设置为 false,目前不支持设置
allow_older_versions
对 ES 的版本兼容性
布尔类型,可设为“true”或者是“false”
output
protocol
数据传输协议
字符串类型,默认为“http”,支持设置为“https”
hosts
索引内网访问地址
数组类型,
如 protocol 选择为“http”,则端口号为80,例如可设置为:[“http://index-xxx.qcloudes.com:80”];
如 protocol 选择为“https”,则端口号为443,如[“https://index-xxx.qcloudes.com:443”]
配置说明
# ============================== Filebeat inputs ===============================

filebeat.inputs:
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /var/log/*.log
# ============================== Filebeat modules ==============================

filebeat.config.modules:
# Glob pattern for configuration loading
path: \\${path.config}/modules.d/*.yml

# Set to true to enable config reloading
reload.enabled: false

# Period on which files under path should be checked for changes
#reload.period: 10s

# ======================= Elasticsearch template setting =======================
setup.template.enabled: false
setup.ilm.enabled: false
#template setting's value is set to false by default. If you set it to true, an error will be reported when the configuration is submitted


# ================================== General ===================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:

# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
# env: staging

# ================================= Processors =================================
processors:
- add_host_metadata:
when.not.contains.tags: forwarded

# ================================== Logging ===================================

# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
#logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publisher", "service".
#logging.selectors: ["*"]
############################# output ######################################
output.elasticsearch:
# Array of hosts to connect to.
allow_older_versions: true
protocol: "http"
hosts: ["索引内网访问地址"]

# Authentication credentials - either API key or username/password.
username: "your index username"
password: "your index password"
indices:
- index: The_index_name
when.equals:
fields.type: log