我知道企业(例如Cloudera )的方式,通过使用CM (通过浏览器)或Cloudera REST API,用户可以访问监控和配置工具。
但是,在没有CM的情况下,如何安排(运行和重新运行) flume代理livecycle,并监控它们的运行/故障状态?在Flume发行版中有这样的东西吗?
发布于 2019-08-28 22:54:26
我尝试在开始时将flume.monitoring.type/port添加到flume-ng中。它完全符合我的需求。
例如,让我们创建一个简单的代理a1。它监听localhost:44444并作为接收器登录到控制台:
# flume.conf
a1.sources = s1
a1.channels = c1
a1.sinks = d1
a1.sources.s1.channels = c1
a1.sources.s1.type = netcat
a1.sources.s1.bind = localhost
a1.sources.s1.port = 44444
a1.sinks.d1.channel = c1
a1.sinks.d1.type = logger
a1.channels.c1.type = memory
a1.channels.c1.capacity = 100
a1.channels.c1.transactionCapacity = 10使用其他参数flume.monitoring.type/port运行它
flume-ng agent -n a1 -c conf -f flume.conf -Dflume.root.logger=INFO,console -Dflume.monitoring.type=http -Dflume.monitoring.port=44123然后在localhost:44123/metrics的浏览器中监控输出
{"CHANNEL.c1":{"ChannelCapacity":"100","ChannelFillPercentage":"0.0","Type":"CHANNEL","EventTakeSuccessCount":"570448","ChannelSize":"0","EventTakeAttemptCount":"570573","StartTime":"1567002601836","EventPutAttemptCount":"570449","EventPutSuccessCount":"570448","StopTime":"0"}}只需尝试一些加载:
dd if=/dev/urandom count=1024 bs=1024 | base64 | nc localhost 44444发布于 2019-08-21 19:22:32
Flume的JSON报告API可用于监控运行状况和性能。Link
https://stackoverflow.com/questions/57555210
复制相似问题