前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >kafka连接异常 原

kafka连接异常 原

作者头像
尚浩宇
发布2018-08-17 11:03:45
1.7K0
发布2018-08-17 11:03:45
举报
文章被收录于专栏:杂烩杂烩

kafka版本是0.10.2.1 本地java客户端版本是0.8.1.1

主要两个错误

第一个是连接拒绝

代码语言:javascript
复制
kafka Connection refused: no further information

然后发现第二个错误

代码语言:javascript
复制
Selector.poll(Selector.java:276) - Error in I/O with localhost/127.0.0.1

怀疑是ip绑定有问题,编辑server.properties,指定ip地址

代码语言:javascript
复制
advertised.host.name=ip地址

重启后,运行客户端,抛出另外一个问题

代码语言:javascript
复制
KafkaException: Failed to construct kafka producer

可能是因为客户端版本过低导致,升级客户端版本后,运行正常。

以下测试demo

生产者

代码语言:javascript
复制
public class ProducerDemo {

	private static final Logger LOG = Logger.getLogger(ProducerDemo.class);

	private static Properties properties = new Properties();

	static {
		properties.put("bootstrap.servers", "172.16.41.151:9092");
		properties.put("acks", "all");
		properties.put("retries", 0);
		properties.put("batch.size", 16384);
		properties.put("linger.ms", 1);
		properties.put("buffer.memory", 33554432);
		properties.put("key.serializer", "org.apache.kafka.common.serialization.ByteArraySerializer");
		properties.put("value.serializer", "org.apache.kafka.common.serialization.ByteArraySerializer");
	}

	public void produce() {
		KafkaProducer<byte[], byte[]> kafkaProducer = new KafkaProducer<byte[], byte[]>(properties);
		ProducerRecord<byte[], byte[]> kafkaRecord = new ProducerRecord<byte[], byte[]>("canal-t_canal",
				"kkk".getBytes(), "ssss".getBytes());
		kafkaProducer.send(kafkaRecord, new Callback() {
			public void onCompletion(RecordMetadata metadata, Exception e) {
				if (null != e) {
					LOG.info("the offset of the send record is {" + metadata.offset() + "}");
					LOG.error(e.getMessage(), e);
				}
				LOG.info("complete!");
			}
		});
		kafkaProducer.close();
	}

	public static void main(String[] args) {
		ProducerDemo kafkaProducerTest = new ProducerDemo();
		for (int i = 0; i < 10; i++) {
			kafkaProducerTest.produce();
		}
	}

}

消费者

代码语言:javascript
复制
public class ConsumerDemo {

	private static final String topic = "canal-t_canal";

	private static final Integer threads = 3;

	public static void main(String[] args) {

		Properties props = new Properties();

		props.put("zookeeper.connect", "172.16.41.151:2181");

		props.put("group.id", "1111");

		props.put("zookeeper.session.timeout.ms", "40000");
		props.put("zookeeper.sync.time.ms", "200");
		props.put("auto.commit.interval.ms", "1000");

		ConsumerConfig config = new ConsumerConfig(props);

		ConsumerConnector consumer = Consumer.createJavaConsumerConnector(config);

		Map<String, Integer> topicCountMap = new HashMap<String, Integer>();

		topicCountMap.put(topic, 1);

		Map<String, List<KafkaStream<byte[], byte[]>>> consumerMap = consumer.createMessageStreams(topicCountMap);

		List<KafkaStream<byte[], byte[]>> streams = consumerMap.get("canal-t_canal");

		for (final KafkaStream<byte[], byte[]> kafkaStream : streams) {

			new Thread(new Runnable() {

				public void run() {

					for (MessageAndMetadata<byte[], byte[]> mm : kafkaStream) {

						String msg = new String(mm.message());

						System.out.println(msg);

					}

				}

			}).start();

		}

	}

}

pom

代码语言:javascript
复制
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>scc-stu</groupId>
	<artifactId>stu-kafka</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>war</packaging>
	<name>stu-kafka</name>
	<description>stu-kafka</description>
	<dependencies>
		<dependency>
			<groupId>org.apache.kafka</groupId>
			<artifactId>kafka-clients</artifactId>
			<version>0.10.0.0</version>
		</dependency>

		<dependency>
			<groupId>org.apache.kafka</groupId>
			<artifactId>kafka_2.11</artifactId>
			<version>0.10.0.0</version>
		</dependency>
		<!-- log4j -->
		<dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.6.1</version>
        </dependency>
	</dependencies>
</project>
本文参与 腾讯云自媒体分享计划,分享自作者个人站点/博客。
原始发表:2017/05/15 ,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体分享计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档