Each Azure integration project needs Azure Service Bus to perform real decoupling between applications.  When interacting with the Service Bus, to write and read messages, you need to have special attention to the reliability.  Loosing a message can potentially have a big business impact.  This blog explains how you can process messages from a Service Bus queue (or topic) in a reliable fashion.

Auto-completing messages

In many tutorials, messages get auto-completed.  Please be aware that this solution is simple, but not reliable at all.  When there is a network interruption between the moment that Service Bus deletes the message and Logic Apps receives the message, your message will be lost.

Peek-lock pattern

In this pattern, a Service Bus message is locked (invisible on the queue) when Logic Apps pulls the message.  Logic Apps needs to explicitly acknowledge (complete) the message, to remove if from the queue.  This ensures that no message can get lost.

Complete at the end

In many implementations, I have seen that the message gets completed at the end of the Logic App.

This solution will work, until something goes wrong.  If the Logic App becomes in a failed state, the Service Bus lock duration (which is 30 seconds by default) will expire and the message will become available on the queue.  As a result, the message will be processed twice.  If it fails again, it will be picked up again until the Service Bus delivery count (which is 10 by default) is reached.  This behavior will be the same if the processing of your Logic App takes longer than the lock duration.  This will happen in many scenarios, because Logic Apps has retry mechanisms on most actions.

Complete at the beginning

That’s why my preferred solution is to complete the message immediately after Logic Apps received it.

This works perfect, until you want to resubmit a message.  The Logic App will fail, because it wants to complete a message that does not exist anymore.

Optimize for resubmit

In order to overcome the last issue, we need to determine if we are dealing with a normal Logic App execution or a resubmit.   In one of my recent posts, I explained that you can determine this via the following expression: empty(trigger()?.sourceHistoryName).  If this expression evaluates false, we don’t need to complete the message.

Conclusion

By implementing this pattern, you don’t have to perform complex try/catch logic, you will not loose any message and you can leverage the out-of-the-box Logic Apps resubmit capabilities.  There are definitely other ways to achieve the same result, but this is my preferred solutions.  If you have other ways to deal with this, please let me know, always interested to learn 🙂

Cheers,
Toon