I think MQ is a genuis solution of non-real-time data requests between systems, especially in distributed systems for e-commerce, finance, big data. Here I am going to document some of my thoughts on MQ.
Most of typical financial process systems, the first level system which directly access by user and submit their financial applications , the financing request goes through several systems will eventually reach to bank center to do payment / settlement – subtract / addition of account money, between each of them, MQ plays a crucial role to ensure the reliability of the entire system via decoupling , traffic peaking, asynchronizing.
In Apache RocketMQ, there are usually three message sending modes we are using:
-
Synchronous send
the
producer.sendMessage()code line wait there til the broker receives the message and returns a confirmation. This mode offers a good reliability. -
Asynchronous send
the
producer.sendMessage()code line won’t be blocked, means I just need to send the message out , and continue to run , don’t block. So there is always a callback, is provided to handle the send result (success or exception) asynchronously. Very applies to High-throughput scenarios where low-latency is needed , but delivery assurance is still important. -
One-way send
just only to send out the message, but i don’t care about whether it received. Typical features : no return , no callback.
-
Ordered messages
If the consumer needs to receive messages as the order that they were sent, usually use the orderd message mode that implemented by sending only 1 message queue in a topic.
-
Delayed message
Send message after a certain time.
-
Transactional message
A bit complex, but it can be a very elegant and ingenious solution for handling time out orders without payment. (will write a essay to record if I got spare time)
The above methods are relatively easy to be implemented, by an everage programmer. I prefer to dive into its design.
This is the figure of Rocket-MQ :
NameServer: think of it as kind of DNS for producers and consumers , brokers register themselves with nameservers, producers query nameservers to find which brokers host which topics, usually we implement multiple nameservers and they don’t need to sync up each other (eventually consistent) .
Broker: imagine as a “post office” , producer send message from left side, consumer receive from right side. Broker persists messages to disk, maintains manages topics and their message queues.
How do they work together ?
1. Broker → Nameserver: "Hi, I'm Broker-A hosting Topic-1, Topic-2"
2. Producer → Nameserver: "Where's Topic-1?"
3. Nameserver → Producer: "Broker-A has it at 192.168.1.10:10911"
4. Producer → Broker-A: Sends messages to Topic-1