首页
学习
活动
专区
工具
TVP
发布
社区首页 >问答首页 >Kafka连接JDBC源OutOfMemoryError

Kafka连接JDBC源OutOfMemoryError
EN

Stack Overflow用户
提问于 2018-05-30 03:09:02
回答 1查看 835关注 0票数 1

我正在使用Confluent JDBC-Source connector运行以下作业

{
    "name": "jobName",
    "config": {
        "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
        "mode": "timestamp",
        "timestamp.column.name": "dateColumn",
        "topic.prefix": "connect-test.",
        "connection.password": "pw",
        "tasks.max": "1",
        "connection.user": "un",
        "poll.interval.ms": "300000",
        "name": "jobName",
        "connection.url": "jdbc:sqlserver://serverName;Database=dbName;user=un;password=pw",
        "table.whitelist": "tableName"
    }
}

我有一个类似的Kafka-Connect作业在相同的数据库和相同的用户上成功运行,但使用另一个较小的表。所以连接不是问题所在。

在运行作业的Kafka-connect服务器上的日志中,我看到了以下内容:

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "KafkaBasedLog Work Thread - connect-configs"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "kafka-coordinator-heartbeat-thread | connect-cluster"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "kafka-producer-network-thread | producer-1"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "KafkaBasedLog Work Thread - connect-offsets"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "kafka-producer-network-thread | producer-3"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "pool-4-thread-1"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "kafka-producer-network-thread | producer-2"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "KafkaBasedLog Work Thread - connect-status"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "DistributedHerder"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "pool-1-thread-5"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "org.eclipse.jetty.server.session.HashSessionManager@2c243a24Timer"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1434297727-21"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1434297727-26"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1434297727-27"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "kafka-producer-network-thread | producer-14"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "kafka-producer-network-thread | producer-15"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "pool-1-thread-4"

所以,没有太多可以继续下去的。运行此作业的服务器现在没有响应,并且不响应REST调用。有什么想法吗?

EN

回答 1

Stack Overflow用户

发布于 2018-05-30 10:05:07

你需要

  1. 通过SSH登录到每个连接工作进程,或者无论您如何到达那里,
  2. 会强制停止Kafka连接进程,使其大于运行Kafka连接的每个服务器上的默认值。例如KAFKA_HEAP_OPTS="-Xms512M -Xmx4G"。假设使用Linux,那么对于kafka用户来说,最好的地方应该是bashrc文件。
  3. 以您为

导出变量的用户身份再次运行connect-distributed

您可能还想导出更多的值,以便能够公开JMX。通过这种方式,您可以在进程崩溃之前对其进行监视

票数 1
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/50591331

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档