In a distributed and disconnected world loosing data is the most terrible thing which can happen. If you are using Mule as ESB and trying to pass messages, how do you ensure you do not loose messages when the system connected at outbound is down? How do you perform retries without asking the publisher to send the data again? Is it hypothetical scenario? Not really. Consider scenario where we have a service endpoint exposed to capture any major actions on a website. These actions are posted to the endpoint and the endpoint sends data to RabbitMQ for another system to pick it up and process the data. These actions do not impact User on the website. So these are silent events being sent out to track the user behavior on the site.
One thing to notice in this flow is what if Rabbit Server goes down? Who is responsible for sending the event again? It should not be the user as user should not even be knowing about the data being sent. Should it be web application? Well why would web application keep track of this and send data again? So someone has to do this again. Why not just build alternate storage within mule flow and send the data back?
So one option to do this is by using a file connector. Below is how the flow may look like:
We have 3 flows going on:
- HttpConnectorSendingToRabbit: This is a public flow exposing the http endpoint for the events to be sent. This flow simply calls another flow SendMsgToRabbit
- SendMsgToRabbit-PrivateFlow: This is private flow which would try to send data to RabbitMQ. In case Rabbit is down, it logs exception, resets the payload and saves the data to file. Note that if any other exception occurs, we don’t want to save the file as this will become never succeeding retry attempt
- FileConnectorSendingToRabbit: This is again a public flow with file connector as inbound. This will monitor the folder where failed requests are saved and retry the request again by calling back SendMsgToRabbit flow.
Although this technique does not guarantee sequencing and may not be the optimal solution, it at least gives a guaranteed delivery of the message by saving the copy of message on the Mule server itself and retrying it.
