文章/答案/技术大牛

发布

社区首页 >问答首页 >我怎样才能让我的卡夫卡经纪人彼此沟通？

问我怎样才能让我的卡夫卡经纪人彼此沟通？
EN

Stack Overflow用户

提问于 2022-10-24 15:54:40

回答 1查看 21关注 0票数 1

我试图创造两个卡夫卡对接者形象互相沟通，但第一个卡夫卡经纪人站起来找不到第二个卡夫卡经纪人。这是我的船坞-复合。Here：

version: '3.8'
services:
  zookeeper:
    image: "bitnami/zookeeper:latest"
    ports:
      - "2181:2181"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - ZOOKEEPER_CLIENT_PORT=2181
      - ZOOKEEPER_TICK_TIME=2000
    tmpfs: "/datalog"
  kafka-1:
    image: "bitnami/kafka:latest"
    container_name: "kafka-1"
    ports:
      - "9094:9094"
    depends_on:
      - zookeeper
    restart: always
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENERS=PLAINTEXT://:9094
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9094
      - KAFKA_DEFAULT_REPLICATION_FACTOR=2
      - KAFKA_NUM_PARTITIONS=2
      - ALLOW_PLAINTEXT_LISTENER=yes
  kafka-2:
    image: "bitnami/kafka:latest"
    container_name: "kafka-2"
    ports:
      - "9095:9095"
    depends_on:
      - zookeeper
    restart: always
    environment:
      - KAFKA_BROKER_ID=2
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENERS=PLAINTEXT://:9095
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9095
      - KAFKA_DEFAULT_REPLICATION_FACTOR=2
      - KAFKA_NUM_PARTITIONS=2
      - ALLOW_PLAINTEXT_LISTENER=yes

卡夫卡容器反复打印出来：

kafka-2                      | [[2022-10-24 15:41:47,131] INFO [Controller id=2, targetBrokerId=1] Client requested connection close from node 1 (org.apache.kafka.clients.NetworkClient)
kafka-2                      | [2022-10-24 15:41:47,232] INFO [Controller id=2, targetBrokerId=1] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)
kafka-2                      | [2022-10-24 15:41:47,232] WARN [Controller id=2, targetBrokerId=1] Connection to node 1 (/127.0.0.1:9094) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka-2                      | [2022-10-24 15:41:47,232] WARN [RequestSendThread controllerId=2] Controller 2's connection to broker 127.0.0.1:9094 (id: 1 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
kafka-2                      | java.io.IOException: Connection to 127.0.0.1:9094 (id: 1 rack: null) failed.
kafka-2                      |  at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
kafka-2                      |  at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:291)
kafka-2                      |  at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:245)
kafka-2                      |  at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)

当我尝试使用kafkajs发送消息时，我会得到错误消息：KafkaJSNonRetriableError: Replication-factor is invalid

编辑：

利用以下建议：

version: '3.8'
services:
  dynamodb-local-express-bp:
    command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
    image: "amazon/dynamodb-local:latest"
    container_name: dynamodb-local-express-bp
    ports:
      - "8000:8000"
    volumes:
      - "./docker/dynamodb:/home/dynamodblocal/data"
    working_dir: /home/dynamodblocal
  zookeeper:
    image: "bitnami/zookeeper:latest"
    ports:
      - "2181:2181"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - ZOOKEEPER_CLIENT_PORT=2181
      - ZOOKEEPER_TICK_TIME=2000
    tmpfs: "/datalog"
  kafka-1:
    image: "bitnami/kafka:latest"
    container_name: "kafka-1"
    expose:
      - 9092
    ports:
      - "9094:9094"
    depends_on:
      - zookeeper
    restart: always
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9094,INTERNAL://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9094,INTERNAL://kafka-1:9092
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,INTERNAL:PLAINTEXT
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
      - KAFKA_CFG_DEFAULT_REPLICATION_FACTOR=2
      - KAFKA_CFG_NUM_PARTITIONS=2
      - ALLOW_PLAINTEXT_LISTENER=yes
  kafka-2:
    image: "bitnami/kafka:latest"
    container_name: "kafka-2"
    ports:
      - "9095:9095"
    depends_on:
      - zookeeper
    restart: always
    environment:
      - KAFKA_BROKER_ID=2
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9095,INTERNAL://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9095,INTERNAL://kafka-1:9092
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,INTERNAL:PLAINTEXT
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
      - KAFKA_CFG_DEFAULT_REPLICATION_FACTOR=2
      - KAFKA_CFG_NUM_PARTITIONS=2
      - ALLOW_PLAINTEXT_LISTENER=yes
      # - KAFKA_BROKER_ID=2
      # - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      # - KAFKA_LISTENERS=PLAINTEXT://:9095
      # - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9095
      # - KAFKA_DEFAULT_REPLICATION_FACTOR=2
      # - KAFKA_NUM_PARTITIONS=2
      # - ALLOW_PLAINTEXT_LISTENER=yes

我收到以下错误消息：

[2022-10-24 18:59:06,286] ERROR [KafkaServer id=2] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka-2                      | java.lang.IllegalArgumentException: requirement failed: Configured end points kafka-1:9092 in advertised listeners are already registered by broker 1
kafka-2                      |  at scala.Predef$.require(Predef.scala:281)
kafka-2                      |  at kafka.server.KafkaServer.$anonfun$createBrokerInfo$3(KafkaServer.scala:512)
kafka-2                      |  at kafka.server.KafkaServer.$anonfun$createBrokerInfo$3$adapted(KafkaServer.scala:510)
kafka-2                      |  at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
kafka-2                      |  at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
kafka-2                      |  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
kafka-2                      |  at kafka.server.KafkaServer.createBrokerInfo(KafkaServer.scala:510)
kafka-2                      |  at kafka.server.KafkaServer.startup(KafkaServer.scala:331)
kafka-2                      |  at kafka.Kafka$.main(Kafka.scala:109)
kafka-2                      |  at kafka.Kafka.main(Kafka.scala)

docker

apache-kafka

dockerfile

回答 1

Stack Overflow用户

回答已采纳

发布于 2022-10-24 16:31:38

这个问题是由网桥码头为您创建的。

这里是一篇优秀的文章，卡夫卡经纪人到底是如何发现对方的。

你需要一个额外的广告听众。127.0.0.1是向您的主机发布广告的那个。你需要一个额外的在码头网络内。

将这些信任添加/更改到您的dockerfile：

kafka-1:
   ...
   expose: 
      - 9092
   environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9094,INTERNAL://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9094,INTERNAL://kafka-1:9092
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,INTERNAL:PLAINTEXT
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME: INTERNAL
      - KAFKA_CFG_DEFAULT_REPLICATION_FACTOR=2
      - KAFKA_CFG_NUM_PARTITIONS=2
      - ALLOW_PLAINTEXT_LISTENER=yes

与kafka-2类似

另外，您使用的是bitnami图像，它们以KAFKA_CFG作为所有Kafka环境变量的前缀。这也是个难题

票数 1

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/74183748

复制

相似问题

问我怎样才能让我的卡夫卡经纪人彼此沟通？
EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问我怎样才能让我的卡夫卡经纪人彼此沟通？EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问我怎样才能让我的卡夫卡经纪人彼此沟通？
EN