前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >连接kafka报错:1 partitions have leader brokers without a matching listener「建议收藏」

连接kafka报错:1 partitions have leader brokers without a matching listener「建议收藏」

作者头像
全栈程序员站长
发布2022-09-13 10:58:41
2K0
发布2022-09-13 10:58:41
举报
文章被收录于专栏:全栈程序员必看

大家好,又见面了,我是你们的朋友全栈君。

服务输出部分错误日志截图

代码语言:javascript
复制
2020/12/25 下午2:32:442020-12-25 14:32:44.320  WARN [tag-service,,,] 1 --- [ntainer#4-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-5, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [tagProcessed-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.321  WARN [tag-service,,,] 1 --- [ntainer#2-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-3, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [filterPendingTasktag-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.327  WARN [tag-service,,,] 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-7, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [entityFreshTags-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.329  WARN [tag-service,,,] 1 --- [ntainer#7-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-2, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [mergeContact-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.421  WARN [tag-service,,,] 1 --- [ntainer#4-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-5, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [tagProcessed-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.422  WARN [tag-service,,,] 1 --- [ntainer#2-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-3, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [filterPendingTasktag-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.427  WARN [tag-service,,,] 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-7, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [entityFreshTags-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.429  WARN [tag-service,,,] 1 --- [ntainer#7-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-2, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [mergeContact-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.522  WARN [tag-service,,,] 1 --- [ntainer#2-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-3, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [filterPendingTasktag-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.524  WARN [tag-service,,,] 1 --- [ntainer#4-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-5, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [tagProcessed-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.528  WARN [tag-service,,,] 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-7, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [entityFreshTags-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.530  WARN [tag-service,,,] 1 --- [ntainer#7-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-2, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [mergeContact-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.623  WARN [tag-service,,,] 1 --- [ntainer#2-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-3, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [filterPendingTasktag-0]
2020/12/25 下午2:32:442020-12-25 14:32:44.625  WARN [tag-service,,,] 1 --- [ntainer#4-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-5, groupId=statistic-processor] 1 partitions have leader brokers without a matching listener, including [tagProcessed-0]

场景:

压测环境

原因:

kafka服务集群中某个节点挂掉了.

kafka部分错误日志

代码语言:javascript
复制
[2020-12-15 18:26:15,970] ERROR Error while appending records to sinkData2Es-0 in dir /data/kafka-logs (kafka.server.LogDirFailureChannel)
java.io.IOException: No space left on device
	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
	at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
	at sun.nio.ch.IOUtil.write(IOUtil.java:65)
	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
	at org.apache.kafka.common.record.MemoryRecords.writeFullyTo(MemoryRecords.java:88)
	at org.apache.kafka.common.record.FileRecords.append(FileRecords.java:164)
	at kafka.log.LogSegment.append(LogSegment.scala:158)
	at kafka.log.Log.$anonfun$append$2(Log.scala:1171)
	at kafka.log.Log.append(Log.scala:2318)
	at kafka.log.Log.appendAsLeader(Log.scala:1002)
	at kafka.cluster.Partition.$anonfun$appendRecordsToLeader$1(Partition.scala:969)
	at kafka.cluster.Partition.appendRecordsToLeader(Partition.scala:957)
	at kafka.server.ReplicaManager.$anonfun$appendToLocalLog$4(ReplicaManager.scala:794)
	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
	at scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:149)
	at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:237)
	at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:230)
	at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:44)
	at scala.collection.mutable.HashMap.foreach(HashMap.scala:149)
	at scala.collection.TraversableLike.map(TraversableLike.scala:238)
	at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
	at scala.collection.AbstractTraversable.map(Traversable.scala:108)
	at kafka.server.ReplicaManager.appendToLocalLog(ReplicaManager.scala:782)
	at kafka.server.ReplicaManager.appendRecords(ReplicaManager.scala:506)
	at kafka.server.KafkaApis.handleProduceRequest(KafkaApis.scala:582)
	at kafka.server.KafkaApis.handle(KafkaApis.scala:128)
	at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:70)
	at java.lang.Thread.run(Thread.java:748)

问题:

怀疑是某个时间段推送数据过多,导致kafka集群(三节点)中一个节点磁盘使用率达到100%,kafka服务停止.

发布者:全栈程序员栈长,转载请注明出处:https://javaforall.cn/153321.html原文链接:https://javaforall.cn

本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档