blocks|key|1654605|text|控制发送到broker之前的时间的参数是linger.ms。默认值为0(无延迟)。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1654606|entityMap^0|K|9|0^^$0|@$1|2|3|4|5|6|7|H|8|@$9|I|A|J|B|C]]|D|@]|E|$]]|$1|F|3|-4|5|6|7|K|8|@]|D|@]|E|$]]]|G|$]]

The parameter that controls the time before sending to broker is <code>linger.ms</code>. Its default value is 0 (no delay).

blocks|key|1869082|text|我在一个完全不同的上下文中得到了这个异常。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1869083|我设置了一个由zookeeper+vm、broker+vm和producer/consumer+vm组成的迷你集群。我打开了服务器(9092)和zookeeper+(2181)上的所有必要端口，然后尝试将来自消费者/发布者vm的消息发布到代理。我得到了OP提到的异常，但由于到目前为止我只发布了一条消息(或者至少我尝试过)，解决方案不可能是增加超时或批处理大小。因此，我搜索并找到了这个邮件列表，它描述了我在尝试使用来自消费者/生产者vm+(ClosedChannelException)的消息时遇到的类似问题：http://grokbase.com/t/kafka/users/152jsjekrm/having-trouble-with-the-simplest-remote-kafka-config，这个邮件列表中的最后一篇文章实际上描述了如何解决这个问题。|offset|length|1869084|长话短说，如果同时遇到ChannelClosedException和Batch+Expired异常，则可能需要在server.config文件中将此行更改为以下内容，然后重新启动代理：|style|CODE|1869085|advertised.host.name=<broker+public+IP+address>|code-block|syntax|javascript|1869086|如果没有设置，它将回退到host.name属性(可能也没有设置)，然后回退到InetAddress+Java类的规范主机名，当然，这最终是不正确的，从而使远程节点感到困惑。|1869087|entityMap|0|LINK|mutability|MUTABLE|url|http://grokbase.com/t/kafka/users/152jsjekrm/having-trouble-with-the-simplest-remote-kafka-config^0|0|74|2P|0|0|B|M|Y|D|1K|D|0|0|C|9|12|B|0^^$0|@$1|2|3|4|5|6|7|Y|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Z|8|@]|9|@$D|10|E|11|1|12]]|A|$]]|$1|F|3|G|5|6|7|13|8|@$D|14|E|15|H|I]|$D|16|E|17|H|I]|$D|18|E|19|H|I]]|9|@]|A|$]]|$1|J|3|K|5|L|7|1A|8|@]|9|@]|A|$M|N]]|$1|O|3|P|5|6|7|1B|8|@$D|1C|E|1D|H|I]|$D|1E|E|1F|H|I]]|9|@]|A|$]]|$1|Q|3|-4|5|6|7|1G|8|@]|9|@]|A|$]]]|R|$S|$5|T|U|V|A|$W|X]]]]

I got this exception in a completely different context.

I have setup a mini cluster of a zookeeper vm, a broker vm and a producer/consumer vm. I opened all neccessary ports on the server (9092) and on the zookeeper (2181) and then tried to publish a message from the consumer/publisher vm to the broker. I got the exception mentioned by the OP, but since I had only published one single message so far (or at least I tried to), the solution couldn't be to increase the timeout or batch size. So I searched on and found this mailing list describing a similar problem I had when trying to consume messages from within the consumer/producer vm (ClosedChannelException): <a href="http://grokbase.com/t/kafka/users/152jsjekrm/having-trouble-with-the-simplest-remote-kafka-config">http://grokbase.com/t/kafka/users/152jsjekrm/having-trouble-with-the-simplest-remote-kafka-config</a>
The last post in this mailing list actually describes how to solve the problem.

Long story short, if you face both the <code>ChannelClosedException</code> and the <code>Batch Expired</code> exception, you likely have to change this line to the following in the <code>server.config</code> file and restart the broker:

<pre><code>advertised.host.name=&lt;broker public IP address&gt;
</code></pre>

If it isn't set, it falls back to the <code>host.name</code> property (which probably isn't set neither) and then falls back to the canonical host name of the <code>InetAddress</code> Java class, which finally isn't correct of course and thus confusing remote nodes.

blocks|key|1654803|text|创建使用者时，将ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG设置为true。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1654804|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

when you create the consumer set ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG to true.

blocks|key|1654864|text|我使用的是Kafka+Java客户端0.11.0.0版。我也开始看到同样的模式，无法始终如一地生成大型消息。它传递的消息很少，还有一些消息失败了。(虽然通过的消息和失败的消息大小相同).In我的情况是，每个消息的大小都在60KB左右，这远远高于Kafka的默认batch.size+16kB，而且我的linger.ms也设置为默认值0。这个错误被抛出，因为生产者客户端在它可以从server.Basically接收到成功响应之前超时，在我的代码中，此调用超时：kafkaProd.send(pr).get()。要解决这个问题，我必须将生产者客户机的默认request.timeout.ms增加到60000|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1654865|entityMap^0|3M|A|46|9|6E|O|7O|I|0^^$0|@$1|2|3|4|5|6|7|H|8|@$9|I|A|J|B|C]|$9|K|A|L|B|C]|$9|M|A|N|B|C]|$9|O|A|P|B|C]]|D|@]|E|$]]|$1|F|3|-4|5|6|7|Q|8|@]|D|@]|E|$]]]|G|$]]

I am using Kafka Java client version 0.11.0.0. I also started seeing the same pattern in failure to produce large messages consistently. It was passing for few of the messages, and failing for some others. (Although both passed and failed messages were of the same size).In my case, each message size was around 60KB, which is far higher than Kafka's default <code>batch.size</code> of 16kB, also my <code>linger.ms</code> was set to default of 0. This error is being thrown as the Producer client is timing out before it can receive a succesful response from the server.Basically, in my code , this call was timing out : <code>kafkaProd.send(pr).get()</code>. To fix this, I had to increase the Producer client's default <code>request.timeout.ms</code> to 60000

blocks|key|1654904|text|在docker-compose中运行的Kafka也有类似的问题。我的docker-compose.yml设置为|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1654905|+KAFKA_ADVERTISED_HOST_NAME:+kafka
+ports:
++++++++-+9092:9092|code-block|syntax|javascript|1654906|但当我试图从外面的码头用骆驼发消息时|1654907|to("kafka:test?brokers=localhost:9092")|1654908|我有一台TimeoutException。我通过添加|1654909|127.0.0.1+kafka|1654910|添加到Windows\System32\drivers\etc\hosts文件，然后将camel+url更改为|1654911|to("kafka:test?brokers=kafka:9092")|1654912|entityMap^0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|V|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|W|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|X|8|@]|9|@]|A|$E|F]]|$1|K|3|L|5|6|7|Y|8|@]|9|@]|A|$]]|$1|M|3|N|5|D|7|Z|8|@]|9|@]|A|$E|F]]|$1|O|3|P|5|6|7|10|8|@]|9|@]|A|$]]|$1|Q|3|R|5|D|7|11|8|@]|9|@]|A|$E|F]]|$1|S|3|-4|5|6|7|12|8|@]|9|@]|A|$]]]|T|$]]

Had a similar issue with Kafka running in a docker-compose.
My docker-compose.yml was set with

<pre><code> KAFKA_ADVERTISED_HOST_NAME: kafka
 ports:
 - 9092:9092
</code></pre>

But when I tried to send a message with camel from outside docker

<pre><code>to("kafka:test?brokers=localhost:9092")
</code></pre>

I got a TimeoutException. I solved it by adding 

<pre><code>127.0.0.1 kafka
</code></pre>

to Windows\System32\drivers\etc\hosts file and then changing my camel url to 

<pre><code>to("kafka:test?brokers=kafka:9092")
</code></pre>

blocks|key|1869428|text|我解决了。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1869429|我的Kafka部署在Docker容器中，容器的网络模式为网桥，主机和容器使用端口映射，我将Kafka服务器的默认端口修改为9102。|1869430|server.properties中用来解决这个问题的配置项有以下两个:+listeners+advertised.listeners|1869431|我尝试了几种组合：|1869432|成功：|1869433|listeners=PLAINTEXT://:9102
advertised.listeners=PLAINTEXT://192.168.0.136:9102|code-block|syntax|javascript|1869434|服务器无法启动：|1869435|listeners=PLAINTEXT://192.168.0.136:9102
advertised.listeners=PLAINTEXT://192.168.0.136:9102|1869436|超时错误：|1869437|listeners=PLAINTEXT://:9102
advertised.listeners=PLAINTEXT://:9102|1869438|entityMap^0|0|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Z|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|10|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|11|8|@]|9|@]|A|$]]|$1|H|3|I|5|6|7|12|8|@]|9|@]|A|$]]|$1|J|3|K|5|L|7|13|8|@]|9|@]|A|$M|N]]|$1|O|3|P|5|6|7|14|8|@]|9|@]|A|$]]|$1|Q|3|R|5|L|7|15|8|@]|9|@]|A|$M|N]]|$1|S|3|T|5|6|7|16|8|@]|9|@]|A|$]]|$1|U|3|V|5|L|7|17|8|@]|9|@]|A|$M|N]]|$1|W|3|-4|5|6|7|18|8|@]|9|@]|A|$]]]|X|$]]

i solved it.
My Kafka is deployed in a Docker container, and the container's network mode is the bridge,The host and container use port mappings,and i I changed the default port to 9102 for the Kafka server.
The configuration items in server.properties to solve the problem are these two:
listeners
advertised.listeners
I tried several combinations:
success：
<pre><code>listeners=PLAINTEXT://:9102
advertised.listeners=PLAINTEXT://192.168.0.136:9102
</code></pre>
server can't start:
<pre><code>listeners=PLAINTEXT://192.168.0.136:9102
advertised.listeners=PLAINTEXT://192.168.0.136:9102
</code></pre>
timeout error:
<pre><code>listeners=PLAINTEXT://:9102
advertised.listeners=PLAINTEXT://:9102
</code></pre>

Using the Apache Kafka Java client (0.9), I'm trying to send a long series of records to the broker using the <a href="https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html">Kafka Producer class</a>. 

The asynchronous <a href="https://kafka.apache.org/090/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html#send(org.apache.kafka.clients.producer.ProducerRecord)">send method</a> returns immediately for a while, then starts blocking on each call for a short time period. After around thirty seconds, the client starts throwing exceptions (<a href="https://kafka.apache.org/090/javadoc/org/apache/kafka/common/errors/TimeoutException.html">TimeoutException</a>), with the message "Batch expired".

What circumstances cause this exception to be thrown?

When does the Apache Kafka client throw a "Batch Expired" exception?

服务器

Windows

容器

Docker

Kafka

Java

使用Apache Kafka Java client (0.9)，我尝试使用向代理发送一长串记录。异步会立即返回一段时间，然后在短时间内开始阻塞每个调用。大约30秒后，客户端开始抛出异常()，并显示消息"Batch expired“。在什么情况下会引发此异常？

问Apache Kafka客户端何时抛出"Batch Expired“异常？
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Apache Kafka客户端何时抛出"Batch Expired“异常？EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Apache Kafka客户端何时抛出"Batch Expired“异常？
EN