Need of messaging queue in Microservices Architecture

In my last article, I wrote about “microservices vs monolithic architecture” which emphasis on the advantage of using microservices architecture, and also focus on the disadvantage of using monolithic architecture. So now you know the advantages of microservices architecture, and how it works, how it can help you in scaling your application, deployment, faster delivery of new features, save you from single point failure.

You are excited to write your new application using microservices architecture. There are certain standards or practices we need to follow to get most out of microservices architecture. And the messaging queue is one of them(eg. rabbit mq, kafka etc.) I would keep writing about remaining practices in next series of articles. But why would I use a messaging queue? what happens if don’t make use of it? We will see how messaging queue makes our life easy for certain scenarios in microservices architecture.

 

Now let’s say you have e-commerce application where users come and buy various items. And they get certain loyalty points or coupon based on their activities like eg. placing an order, giving a review on purchased item etc. Every 6 months, and you want to checks past activity stream of users and update their membership level, loyalty points or coupon. Now you need to propose a solution which can send these details to users. Let’s assume there are two services here, the first one which calculates these attributes, and the second which basically send the notification to the end users via mail, SMS, push notification. So you get huge number of requests to your first services, which might be processing it with many number of threads. And after that, you would want to touch messaging service which takes care of sending the notification to end users.

There are two ways you can handle it, the first one is rest synchronous model, where you make an external call to messaging service and wait for the response. the second one is messaging based asynchronous model.

In the first approach, there are considerable problems like you might not able to respond in very short time as there could be a huge amount of requests, and for each request, you are making external call synchronously, it blocks current thread too until it gets a response from external service. There might be cases where external service is down. You are not sure on external dependency of messaging service. It might cause a delay in response. At the same time if we use the second approach, which basically pushes these changes/events to queue/topic and, respond back to the users. And it also allows multiple receivers to process these messages asynchronously, which can help you out in terms of good performance. If you run it on multiple nodes, could give you result in terms of high availability. So once you push message to the queue, your job is done. Messaging queue will take care of rest. And you are not blocking anything here. So consumers/receivers can read messages from the queue and, process it asynchronously. There could be more examples of using messaging based model.

We will see few more use cases of using queue-based asynchronous model, where basically you are pushing change events to other microservices to maintain consistency. Let say you are working with catalogue team, which takes care of products uploaded by merchant side and mapping it to the clean schema of products in your application. When you are done with mapping, you need to send out this mapping event to other services eg. search service which gives you mapped product based on this. Once search receives this, it can update its products so that when next time user search it should give correct result specific to the mapped product. Let’s assume you got some changes in the attribute of a given product in product service. You might want to push these changes to other microservices which basically make use of some logic which is based on products. And if you look at these two examples, you would not want to block your current thread to process this. Let’s consider one more use cases where you would want to get a response from the second service too. As messaging queue provides one-way communication, so you would need to have one more queue in opposite direction.

So that’s how messaging based asynchronous model can make your life easy in certain scenarios eg. when external service is down, high latency, availability, sending change events to multiple microservices which can seriously affect your application performance. It also provides loose coupling between microservices.