Relevant Concepts
Timed message : After a message is sent to the server, the actual business does not want the consumer to receive the message immediately. Instead, it postpones it to a certain time point for consumption. Such messages are collectively called timed messages.
Delayed message : After a message is sent to the server, the actual business does not want the consumer to receive the message immediately. Instead, it postpones it by a certain time period for consumption. Such messages are collectively called delayed messages.
In fact, scheduled messages can be seen as a special usage of delayed messages, and their ultimate implementation effect is consistent with delayed messages.
Applicable Scenario
If the system is a monolithic architecture, implementing delays through business code or using third-party components makes little difference. However, as the architecture becomes more complex, forming a large-scale distributed system with dozens or even hundreds of microservices, and implementing timing logic within the application can lead to various issues. Once a node running the delay program encounters a problem, the entire delayed logic will be affected.
To address these issues, leveraging the characteristics of delayed messages by delivering them to message queues is a better solution. This approach allows for unified calculation of delay times, while retry and dead-letter mechanisms ensure that messages are not lost.
Below are specific scenario examples:
After sending a WeChat red packet, the producer sends a message delayed by 24 hours. When the consumer program receives the message after 24 hours, it checks whether the user has claimed the red packet. If not, it returns the red packet to the original account.
After an order is placed on a mini-program for a certain item, the backend stores a message delayed by 30 minutes. When the consumer receives the message after the specified time, it triggers a check on the payment status. If payment has not been made, the order is canceled, thus implementing logic to cancel orders if payment is not completed within 30 minutes.
After a user sets a message as a to-do item on WeChat, he or she can also send a timed message. The server actively consumes this timed message to remind the user of the to-do item.
Product Use
The TDMQ for Pulsar SDK provides a dedicated API to implement timed messages and delayed messages.
For scheduled messages, you need to provide the moment when the message should be sent.
For delayed messages, you need to provide a duration of time as the delay length.
Scheduled Messages
Scheduled messages are implemented using the deliverAt() method of the
producer. Sample code:String value = "message content";try {//It is necessary to explicitly convert the desired time to a Timestamp.long timeStamp = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse("2020-11-11 00:00:00").getTime();//Use the deliverAt method of the producer to implement scheduled messages.MessageId msgId = producer.newMessage().value(value.getBytes()).deliverAt(timeStamp).send();} catch (ParseException e) {// TODO Add a method to handle Timestamp parsing failure.e.printStackTrace();}
Note
The time range for scheduled messages is calculated from the current time, allowing for any moment within the next 864,000 seconds (10 days). For example, if it starts at 12:00 on October 1st, it can be scheduled for up to 12:00 on October 11th.
Timed messages cannot be sent using batch mode. Set the
enableBatch parameter to false when you create the producer.The consumption mode of timed messages only supports consumption in shared mode. Otherwise, the timing effect is lost (key-shared is not supported either).
Delayed Messages
Delayed messages are implemented using the deliverAfter() method of the
producer. Below is an example code snippet:String value = "message content";//You need to specify the duration of the delay.long delayTime = 10L;//Use the deliverAfter method of the producer to implement delayed messages.MessageId msgId = producer.newMessage().value(value.getBytes()).deliverAfter(delayTime, TimeUnit.SECONDS) //Units can be freely selected..send();
Note
The duration range for delayed messages is from 0 to 864,000 seconds (0 seconds to 10 days). For example, if it starts at 12:00 on October 1, it can be scheduled for a maximum of 864,000 seconds.
When the delay time of a delayed message exceeds 10 days in the Go SDK, duplicate messages may occur.
Delayed messages cannot be sent using batching mode. Set the
enableBatching parameter to false when you create the producer.The consumption mode of delayed messages only supports consumption in shared mode. Otherwise, the timing effect is lost (key-shared is not supported either).
Instructions and Limitations
When you use timed or delayed messages, it is recommended to use a different topic to manage them from normal messages. That is, send timed and delayed messages to a fixed topic, and send normal messages to another topic. This facilitates subsequent management and maintenance and increases stability.
When using both types of messages, ensure that the client machine's clock and the server machine's clock (all regions are in UTC+8 Beijing time) are synchronized to avoid time differences.
The deviation between scheduled messages and delayed messages is accurate to about 1s.
Timed and delayed messages are not supported in batch mode (batch sending). Batching can lead to message accumulation. As a precaution, set the
enableBatching parameter to false when you create the producer.The consumption mode of timed and delayed messages only supports consumption using the shared mode. Otherwise, the timing or delay effect is lost (key-shared is not supported either).
Regarding the time range for scheduled and delayed messages, the maximum is 10 days.
When scheduled messages are used, the scheduled time must be later than the current time, otherwise, messages will be sent to consumers immediately.
After the timing is set, the TTL time (the maximum retention time) is still calculated from the time the message is sent. For example, if the message is timed to be sent in 2 days, and TTL of the message is set to 1 day, the message is deleted after 1 day. At this time, you should make sure that the TTL time is greater than the delay time. That is, the TTL is set to be greater than or equal to 2 days. Otherwise, the message is deleted when the TTL expires. The same applies to delayed messages.
Normal topics support the sending and receiving of timed/delayed messages. You can call the API in Product Use to send timed/delayed messages.