The content of this page has been automatically translated by AI. If you encounter any problems while reading, you can view the corresponding content in Chinese.
This article mainly introduces the method of switching the service of a self-built RabbitMQ cluster to Tencent Cloud Message Queue RabbitMQ using the dual write and dual consumption scheme.
Scheme: Dual Write and Dual Consumption Mode
Prerequisites
1. Purchased a RabbitMQ instance on the cloud
2. Migrated the metadata of the self-built RabbitMQ cluster to Tencent Cloud RabbitMQ.
Operation step
1. Modify the access information of certain nodes in the consumer cluster to connect corresponding consumers to the new RabbitMQ cluster. They will consume messages in the new cluster, while the rest will continue to consume messages in the old cluster.
2. Modify the access information of certain nodes in the producer cluster to connect corresponding producers to the new RabbitMQ cluster. They will send messages to the new cluster, while the rest will continue to send messages to the old cluster. To avoid message repetition or loss, implement the idempotency logic for message consumption in advance.
3. Connect the remaining producers to the new RabbitMQ cluster. Then, all messages will be sent to the new RabbitMQ cluster.
Tips 1: You can confirm that the production traffic of the self-built cluster has stopped on the RabbitMQ community management console of the self-built cluster.
Tips 2: You can confirm that the backlog of messages in the self-built RabbitMQ cluster is decreasing on the RabbitMQ community management console of the self-built cluster.
4. Check whether there are heaped messages that are not consumed in the old RabbitMQ cluster, and if not, connect the remaining consumers to the new RabbitMQ cluster. At this point, the migration is completed.
Tips: Confirm the message writing and consumption status in the cloud RabbitMQ cluster and ensure that there are no heaped messages.
Note:
If you don't follow the above steps strictly, for example, if you switch producers first and then consumers, message loss may occur.
Before switching the remaining consumers, make sure that all messages in the old RabbitMQ cluster have been consumed; otherwise, some messages may not be consumed.
Possible Issues
Order issues
Message sequence cannot be guaranteed due to cluster switch.
Message repetition
In theory, it won’t repeat messages, but in extreme cases, it might happen. For example, during the switch, a consumer consumes a message but has not yet acknowledged (ACK) it to the server (old RabbitMQ cluster), leading the message to enter the retry queue and causing duplicate consumption. Implementing idempotency logic for message processing can avoid this issue.
Consumption delays
During the read switch, partitions are reallocated with rebalancing between the queue and consumer client. This may cause a short consumption delay, but you don't need to handle it because it won’t persist after the switch.
