前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >Rocketmq之No route info of this topic解决思路

Rocketmq之No route info of this topic解决思路

作者头像
lyb-geek
发布2019-06-18 10:52:23
44.9K1
发布2019-06-18 10:52:23
举报
文章被收录于专栏:Linyb极客之路Linyb极客之路

前言

用过rocketmq的人,采用客户端调用的时候,可能会相对高频的出现的 No route info of this topic这个异常问题,然后你可能会拿着这个问题,直接谷歌百度一把,会发现网上告诉你的答案,可能会有如下几种

1、配置autoCreateTopicEnable=true,如果这个属性没有配置,且你没有手动创建topic,就会出现上面的异常

注:这个属性在高版本已经默认配置了

2、fastjson版本太低

3、防火墙问题

...

最后你可能试了他们提供的方法,发现,坑爹呢,一点用都没有,后边谷歌百度无门了,你只好去官网溜达溜达,看能不能捡到宝,接着你在官网的FAQ你会找到如下相关的问题以及答案回复

Producer complains “No Topic Route Info”, how to diagnose?

代码语言:javascript
复制
Frequently Asked Questions
The following questions are frequently asked with regard to the RocketMQ project in general.

General
1. Why did we create rocketmq project instead of selecting other products?
Please refer to Why RocketMQ

2. Do I have to install other softewares, such as zookeeper, to use RocketMQ?
No. RocketMQ can run independently.

Usage
1. Where does the newly created Consumer ID start consuming messages?
If the topic sends a message within three days, then the consumer start consuming messages from the first message saved in the server.
If the topic sends a message three days ago, the consumer starts to consume messages from the latest message in the server, in other words, starting from the tail of message queue.
If such consumer is rebooted, then it starts to consume messages from the last consumption location.
2. How to reconsume message when consumption fails?
Cluster consumption pattern The consumer business logic code returns Action.ReconsumerLater, NULL, or throws an exception, if a message failed to be consumed, it will retry for up to 16 times, after that, the message would be descarded.

Broadcast consumption pattern The broadcaset consumption still ensures that a message is consumered at least once, but no resend option is provided.

3. How to query the failed message if there is a consumption failure?
Using topic query by time, you can query messages within a period of time.
Using Topic and Message Id to accurately query the message.
Using Topic and Message Key accurately query a class of messages with the same Message Key.
4. Are messages delivered exactly once?
RocketMQ ensures that all messages are delivered at least once. In most cases, the messages are not repeated.

5. How to add a new broker?
Start up a new broker and register it to the same list of name servers.
By default, only internal system topics and consumer groups are created automatically. If you would like to have your business topic and consumer groups on the new node, please replicate them from the existing broker. Admin tool and command lines are provided to handle this.
Configuration related
The following answers are all default values and can be modified by configuration.

1. How long are the messages saved on the server?
Stored messages are will be saved for up to 3 days, and messages that are not consumed for more than 3 days will be deleted.

2. What is the size limit for message Body?
Generally 256KB.

3. How to set the number of consumer threads?
When you start Consumer, set a ConsumeThreadNums property, example is as follows:

consumer.setConsumeThreadMin(20);
consumer.setConsumeThreadMax(20);
Errors
1. If you start a producer or consumer failed and the error message is producer group or consumer repeat?
Reason:Using the same Producer /Consumer Group to launch multiple instances of Producer/Consumer in the same JVM may cause the client fail to start.

Solution: Make sure that a JVM corresponding to one Producer /Consumer Group starts only with one Producer/Consumer instance.

2. If consumer failed to start loading json file in broadcast mode?
Reason: Fastjson version is too low to allow the broadcast consumer to load local offsets.json, causing the consumer boot failure. Damaged fastjson file can also cause the same problem.

Solution: Fastjson version has to be upgraded to rocketmq client dependent version to ensure that the local offsets.json can be loaded. By default offsets.json file is in /home/{user}/.rocketmq_offsets. Or check the integrity of fastjson.

3. What is the impact of a broker crash?
Master crashes
Messages can no longer be sent to this broker set, but if you have another broker set available, messages can still be sent given the topic is present. Messages can still be consumed from slaves.

Some slave crash
As long as there is another working slave, there will be no impact on sending messages. There will also be no impact on consuming messages except when the consumer group is set to consume from this slave preferably. By default, comsumer group consumes from master.

All slaves crash
There will be no impact on sending messages to master, but, if the master is SYNC_MASTER, producer will get a SLAVE_NOT_AVAILABLE indicating that the message is not sent to any slaves. There will also be no impact on consuming messages except that if the consumer group is set to consume from slave preferably. By default, comsumer group consumes from master.

4. Producer complains “No Topic Route Info”, how to diagnose?
This happens when you are trying to send messages to a topic whose routing info is not available to the producer.

Make sure that the producer can connect to a name server and is capable of fetching routing meta info from it.
Make sure that name servers do contain routing meta info of the topic. You may query the routing meta info from name server through topicRoute using admin tools or web console.
Make sure that your brokers are sending heartbeats to the same list of name servers your producer is connecting to.
Make sure that the topic’s permssion is 6(rw-), or at least 2(-w-).
If you can’t find this topic, create it on a broker via admin tools command updateTopic or web console.

嗯,很好,问题和咱们的差不多一样,还有提供答案,肯定能解决,好的,马上行动,结果又坑爹了,依然没有效果

被坑之后,开始静下心来重新思考问题,下边是具体的解决方法

1、日志大法

1、查看broker日志

a、关注broker是否有注册到nameserver

代码语言:javascript
复制
register broker to name server localhost:9876 OK

b、关注生产者是否连接到broker

代码语言:javascript
复制
new producer connected, group: msgProduce channel: ClientChannelInfo [channel=[id: 0x13c55e87, L:/127.0.0.1:10911 - R:/127.0.0.1:59916], clientId=127.0.0.1@Producer-135915d0c39-dbdb-46a4-b75c-efe368b718fe, language=JAVA, version=252, lastUpdateTimestamp=1548498274245]

c、查看已经创建的topic是否包含自己想要的topic

代码语言:javascript
复制
2019-01-26 17:03:31 INFO main - load exist local topic, TopicConfig [topicName=messageTopic, readQueueNums=8, writeQueueNums=8, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false]
2019-01-26 17:03:31 INFO main - load exist local topic, TopicConfig [topicName=interTopic, readQueueNums=8, writeQueueNums=8, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false]
2019-01-26 17:03:31 INFO main - load exist local topic, TopicConfig [topicName=TopicTest, readQueueNums=4, writeQueueNums=4, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false]
2019-01-26 17:03:31 INFO main - load exist local topic, TopicConfig [topicName=broker-a, readQueueNums=1, writeQueueNums=1, perm=RWX, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false]

d、查看消费者是否连接到broker

代码语言:javascript
复制
new consumer connected, group: msgConsumer CONSUME_PASSIVELY CLUSTERING channel: ClientChannelInfo [channel=[id: 0x4d6d00e6, L:/127.0.0.1:10911 - R:/127.0.0.1:59928], clientId=127.0.0.1@8b8ac95a-9514-43b0-abfa-6b1dd9923b5e, language=JAVA, version=252, lastUpdateTimestamp=1548498275493]

2、查看nameserver日志

和broker大同小异

比如查看topic

代码语言:javascript
复制
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, messageTopic QueueData [brokerName=broker-a, readQueueNums=8, writeQueueNums=8, perm=6, topicSynFlag=0]
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, interTopic QueueData [brokerName=broker-a, readQueueNums=8, writeQueueNums=8, perm=6, topicSynFlag=0]
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, TopicTest QueueData [brokerName=broker-a, readQueueNums=4, writeQueueNums=4, perm=6, topicSynFlag=0]
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, broker-a QueueData [brokerName=broker-a, readQueueNums=1, writeQueueNums=1, perm=7, topicSynFlag=0]
2019-01-26 17:03:32 INFO RemotingExecutorThread_1 - new topic registered, %RETRY%message10-35-51-82 QueueData [brokerName=broker-a, readQueueNums=1, writeQueueNums=1, perm=6, topicSynFlag=0]

比如关注broker是否注册到nameserver

代码语言:javascript
复制
new broker registered, localhost:10911 HAServer: localhost:10912

3、创建topic

a、在shell里面执行如下

代码语言:javascript
复制
sh mqadmin updateTopic -blocalhost:10911 -nlocalhost:9876 -tinterTopic

3、创建topic

b、通过可视化web控制台rocketmq-console进行创建,其控制台下载地址如下

https://github.com/apache/rocketmq-externals

当你发现nameserver,broker,topic都已经正常的情况下,却仍然报上面的异常,这种情况下,你可以试下第二种方法

2、代码调试跟踪大法

对于上面的异常,你调试的断点可以设置在如下

1、org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl中的 TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic());

2、org.apache.rocketmq.client.impl.factory.MQClientInstance中的 public TopicRouteData getTopicRouteInfoFromNameServer方法

3、项目组的出现那个异常的最后排查出来的原因

netty包冲突了

冲突的原因是项目采用的rocketmq的客户端是4.2版本,而项目组采用的springcloud,其eureka客户端引用的netty包是4.0版本,这个问题其实也隐藏很深,因为业务项目组是没有自己引入netty包,除了rocketmq客户端,这个是属于jar包依赖传递问题

4、最后解决方案

去除冲突的netty包,去除后成功消费

5、附录依赖传递的图

总结

这个问题前前后后大概花了4左右天的时间,其中百度谷歌花了3天半的时间,最后静下心排查,通过日志,源码跟踪,花了不到半天的时间解决。这次的排查给我的感受的网上的答案可能五花八门,本来是为了节约时间,采用拿来主义的方法,结果适得其反。。。

本文参与 腾讯云自媒体分享计划,分享自微信公众号。
原始发表:2019-06-08,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 Linyb极客之路 微信公众号,前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体分享计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 前言
    • 1、日志大法
    • 2、代码调试跟踪大法
    • 3、项目组的出现那个异常的最后排查出来的原因
    • 4、最后解决方案
    • 5、附录依赖传递的图
    • 总结
    领券
    问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档