专栏首页Linyb极客之路Rocketmq之No route info of this topic解决思路

Rocketmq之No route info of this topic解决思路

前言

用过rocketmq的人,采用客户端调用的时候,可能会相对高频的出现的 No route info of this topic这个异常问题,然后你可能会拿着这个问题,直接谷歌百度一把,会发现网上告诉你的答案,可能会有如下几种

1、配置autoCreateTopicEnable=true,如果这个属性没有配置,且你没有手动创建topic,就会出现上面的异常

注:这个属性在高版本已经默认配置了

2、fastjson版本太低

3、防火墙问题

...

最后你可能试了他们提供的方法,发现,坑爹呢,一点用都没有,后边谷歌百度无门了,你只好去官网溜达溜达,看能不能捡到宝,接着你在官网的FAQ你会找到如下相关的问题以及答案回复

Producer complains “No Topic Route Info”, how to diagnose?

Frequently Asked Questions
The following questions are frequently asked with regard to the RocketMQ project in general.

General
1. Why did we create rocketmq project instead of selecting other products?
Please refer to Why RocketMQ

2. Do I have to install other softewares, such as zookeeper, to use RocketMQ?
No. RocketMQ can run independently.

Usage
1. Where does the newly created Consumer ID start consuming messages?
If the topic sends a message within three days, then the consumer start consuming messages from the first message saved in the server.
If the topic sends a message three days ago, the consumer starts to consume messages from the latest message in the server, in other words, starting from the tail of message queue.
If such consumer is rebooted, then it starts to consume messages from the last consumption location.
2. How to reconsume message when consumption fails?
Cluster consumption pattern The consumer business logic code returns Action.ReconsumerLater, NULL, or throws an exception, if a message failed to be consumed, it will retry for up to 16 times, after that, the message would be descarded.

Broadcast consumption pattern The broadcaset consumption still ensures that a message is consumered at least once, but no resend option is provided.

3. How to query the failed message if there is a consumption failure?
Using topic query by time, you can query messages within a period of time.
Using Topic and Message Id to accurately query the message.
Using Topic and Message Key accurately query a class of messages with the same Message Key.
4. Are messages delivered exactly once?
RocketMQ ensures that all messages are delivered at least once. In most cases, the messages are not repeated.

5. How to add a new broker?
Start up a new broker and register it to the same list of name servers.
By default, only internal system topics and consumer groups are created automatically. If you would like to have your business topic and consumer groups on the new node, please replicate them from the existing broker. Admin tool and command lines are provided to handle this.
Configuration related
The following answers are all default values and can be modified by configuration.

1. How long are the messages saved on the server?
Stored messages are will be saved for up to 3 days, and messages that are not consumed for more than 3 days will be deleted.

2. What is the size limit for message Body?
Generally 256KB.

3. How to set the number of consumer threads?
When you start Consumer, set a ConsumeThreadNums property, example is as follows:

consumer.setConsumeThreadMin(20);
consumer.setConsumeThreadMax(20);
Errors
1. If you start a producer or consumer failed and the error message is producer group or consumer repeat?
Reason:Using the same Producer /Consumer Group to launch multiple instances of Producer/Consumer in the same JVM may cause the client fail to start.

Solution: Make sure that a JVM corresponding to one Producer /Consumer Group starts only with one Producer/Consumer instance.

2. If consumer failed to start loading json file in broadcast mode?
Reason: Fastjson version is too low to allow the broadcast consumer to load local offsets.json, causing the consumer boot failure. Damaged fastjson file can also cause the same problem.

Solution: Fastjson version has to be upgraded to rocketmq client dependent version to ensure that the local offsets.json can be loaded. By default offsets.json file is in /home/{user}/.rocketmq_offsets. Or check the integrity of fastjson.

3. What is the impact of a broker crash?
Master crashes
Messages can no longer be sent to this broker set, but if you have another broker set available, messages can still be sent given the topic is present. Messages can still be consumed from slaves.

Some slave crash
As long as there is another working slave, there will be no impact on sending messages. There will also be no impact on consuming messages except when the consumer group is set to consume from this slave preferably. By default, comsumer group consumes from master.

All slaves crash
There will be no impact on sending messages to master, but, if the master is SYNC_MASTER, producer will get a SLAVE_NOT_AVAILABLE indicating that the message is not sent to any slaves. There will also be no impact on consuming messages except that if the consumer group is set to consume from slave preferably. By default, comsumer group consumes from master.

4. Producer complains “No Topic Route Info”, how to diagnose?
This happens when you are trying to send messages to a topic whose routing info is not available to the producer.

Make sure that the producer can connect to a name server and is capable of fetching routing meta info from it.
Make sure that name servers do contain routing meta info of the topic. You may query the routing meta info from name server through topicRoute using admin tools or web console.
Make sure that your brokers are sending heartbeats to the same list of name servers your producer is connecting to.
Make sure that the topic’s permssion is 6(rw-), or at least 2(-w-).
If you can’t find this topic, create it on a broker via admin tools command updateTopic or web console.

嗯,很好,问题和咱们的差不多一样,还有提供答案,肯定能解决,好的,马上行动,结果又坑爹了,依然没有效果

被坑之后,开始静下心来重新思考问题,下边是具体的解决方法

1、日志大法

1、查看broker日志

a、关注broker是否有注册到nameserver

register broker to name server localhost:9876 OK

b、关注生产者是否连接到broker

new producer connected, group: msgProduce channel: ClientChannelInfo [channel=[id: 0x13c55e87, L:/127.0.0.1:10911 - R:/127.0.0.1:59916], clientId=127.0.0.1@Producer-135915d0c39-dbdb-46a4-b75c-efe368b718fe, language=JAVA, version=252, lastUpdateTimestamp=1548498274245]

c、查看已经创建的topic是否包含自己想要的topic

2019-01-26 17:03:31 INFO main - load exist local topic, TopicConfig [topicName=messageTopic, readQueueNums=8, writeQueueNums=8, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false]
2019-01-26 17:03:31 INFO main - load exist local topic, TopicConfig [topicName=interTopic, readQueueNums=8, writeQueueNums=8, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false]
2019-01-26 17:03:31 INFO main - load exist local topic, TopicConfig [topicName=TopicTest, readQueueNums=4, writeQueueNums=4, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false]
2019-01-26 17:03:31 INFO main - load exist local topic, TopicConfig [topicName=broker-a, readQueueNums=1, writeQueueNums=1, perm=RWX, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false]

d、查看消费者是否连接到broker

new consumer connected, group: msgConsumer CONSUME_PASSIVELY CLUSTERING channel: ClientChannelInfo [channel=[id: 0x4d6d00e6, L:/127.0.0.1:10911 - R:/127.0.0.1:59928], clientId=127.0.0.1@8b8ac95a-9514-43b0-abfa-6b1dd9923b5e, language=JAVA, version=252, lastUpdateTimestamp=1548498275493]

2、查看nameserver日志

和broker大同小异

比如查看topic

2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, messageTopic QueueData [brokerName=broker-a, readQueueNums=8, writeQueueNums=8, perm=6, topicSynFlag=0]
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, interTopic QueueData [brokerName=broker-a, readQueueNums=8, writeQueueNums=8, perm=6, topicSynFlag=0]
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, TopicTest QueueData [brokerName=broker-a, readQueueNums=4, writeQueueNums=4, perm=6, topicSynFlag=0]
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, broker-a QueueData [brokerName=broker-a, readQueueNums=1, writeQueueNums=1, perm=7, topicSynFlag=0]
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, %RETRY%message10-35-51-82 QueueData [brokerName=broker-a, readQueueNums=1, writeQueueNums=1, perm=6, topicSynFlag=0]

比如关注broker是否注册到nameserver

new broker registered, localhost:10911 HAServer: localhost:10912

3、创建topic

a、在shell里面执行如下

sh mqadmin updateTopic -blocalhost:10911 -nlocalhost:9876 -tinterTopic

3、创建topic

b、通过可视化web控制台rocketmq-console进行创建,其控制台下载地址如下

https://github.com/apache/rocketmq-externals

当你发现nameserver,broker,topic都已经正常的情况下,却仍然报上面的异常,这种情况下,你可以试下第二种方法

2、代码调试跟踪大法

对于上面的异常,你调试的断点可以设置在如下

1、org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl中的 TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic());

2、org.apache.rocketmq.client.impl.factory.MQClientInstance中的 public TopicRouteData getTopicRouteInfoFromNameServer方法

3、项目组的出现那个异常的最后排查出来的原因

netty包冲突了

冲突的原因是项目采用的rocketmq的客户端是4.2版本,而项目组采用的springcloud,其eureka客户端引用的netty包是4.0版本,这个问题其实也隐藏很深,因为业务项目组是没有自己引入netty包,除了rocketmq客户端,这个是属于jar包依赖传递问题

4、最后解决方案

去除冲突的netty包,去除后成功消费

5、附录依赖传递的图

总结

这个问题前前后后大概花了4左右天的时间,其中百度谷歌花了3天半的时间,最后静下心排查,通过日志,源码跟踪,花了不到半天的时间解决。这次的排查给我的感受的网上的答案可能五花八门,本来是为了节约时间,采用拿来主义的方法,结果适得其反。。。

本文分享自微信公众号 - Linyb极客之路(gh_c420b2cf6b47),作者:linyb极客之路

原文出处及转载信息见文内详细说明,如有侵权,请联系 yunjia_community@tencent.com 删除。

原始发表时间:2019-06-08

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • @Scheduled注解各参数详解

    该参数接收一个cron表达式,cron表达式是一个字符串,字符串以5或6个空格隔开,分开共6或7个域,每一个域代表一个含义。

    lyb-geek
  • 微服务网关基础组件 - zuul入门

    github地址:https://github.com/Netflix/zuul/

    lyb-geek
  • Mysql慢查询日志的使用 和 Mysql的优化

    1、临时开启慢查询日志(如果需要长时间开启,则需要更改mysql配置文件,第6点有介绍)

    lyb-geek
  • Kernel to User land: APC injection

    When running in Kernel mode, it may be necessary to inject code into a User-land...

    战神伽罗
  • XML进阶:Level 1 - XML简介

    Woodson
  • 解决windows7不能设置壁纸的问题

    我的系统是windows server 2008 r2(启用windows 7桌面体验)最近因为系统慢的缘故,强关了一下机,开机进入系统,发现成黑屏了,改变wi...

    williamwong
  • 将机器学习并行化为最终用户服务(cs.DC)

    随着 ML 应用程序变得越来越普遍,经过全面培训的系统越来越多地提供给广大公众,允许最终用户使用自己的数据提交查询,并高效地检索结果。随着此类服务的日益成熟,新...

    Donuts_choco
  • 搭建Ubuntu12.04交叉编译服务器

    最近学着搭建一台Linux服务器作为交叉编译的主机之用,服务器端选择了Ubuntu12.04 LTS桌面版,客户端采用Windows XP平台,使用SSH工具...

    ccf19881030
  • 【论文推荐】最新六篇视觉问答(VQA)相关论文—盲人问题、物体计数、多模态解释、视觉关系、对抗性网络、对偶循环注意力

    【导读】专知内容组整理了最近六篇视觉问答(Visual Question Answering)相关文章,为大家进行介绍,欢迎查看! 1. VizWiz Gran...

    WZEARW
  • 【转载】使用python库--Graphviz为论文画出漂亮的示意图

    DOT is a plain text graph description language. It is a simple way of describing...

    marsggbo

扫码关注云+社区

领取腾讯云代金券