[2017-10-12 16:52:38,141] ERROR Processor got uncaught exception. (kafka.network.Processor)
java.lang.ArrayIndexOutOfBoundsException: 18
at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:68)
at org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:79)
at kafka.network.Processor$$anonfun$run$11.apply(SocketServer.scala:426)
at kafka.network.Processor$$anonfun$run$11.apply(SocketServer.scala:421)
at scala.collection.Iterator$class.foreach(Iterator.scala:742)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
SocketServer.scala
里面, 大家有兴趣可以自行查阅 ArrayIndexOutOfBoundsException
异常已经被正确处理, 也找到了相应的业务来源;Group "xxx" coordinator is xxx.xxx.xxx.xxx:9092 id 3
Broker: Not coordinator for group
Not coordinator
, 对应broker端是说虽然coordinator确认了,但是没有在这个 coodinator上找到这个group对应的metada信息;Loading
;Loading offsets and group metadata from [__consumer_offsets,19] (kafka.coordinator.GroupMetadataManager)
Finished loading offsets from [__consumer_offsets,27] in 1205 milliseconds. (kafka.coordinator.GroupMetadataManager)
也没有发生任何的exception的日志
1. 去到\_\_conuser\_offsets partition相应的磁盘目录查看,发生有2000多个log文件, 每个在100M左右;
2. kaka 的log compac功能失效了, 这个问题在之前的文章里有过介绍: [Kafka运维填坑](https://www.jianshu.com/p/d2cbaae38014),
3. log compact相关介绍可以参考 [Kafka的日志清理-LogCleaner](https://www.jianshu.com/p/3e0d426313aa)
1. 即使log cleaner功能失败, 为了加速loading, 我们手动删除了大部分的log文件; 这样作有一定风险, 可能会导致某些group的group metadata和committed offset丢失, 从而触发客户端在消费时offset reset;