Understanding the transactional behavior of an Azure Message Queue

As both Udi and Ayende mention, a remote queueing system (like azure queue storage) isn’t really comparable to a local queueing system (such as msmq).

One of the main differences is how they behave transactionally. Well in fact the remote queueing systems aren’t transactional at all, but support a timeout mechanism  to simulate transactions (often called queue-peek-lock mechanism).

The queue-peek-lock mechanism works in the following way:

  • Once a message is picked up by a process, the message will stay in the queue but will become invisible to other processes.
  • When the process is finished handling the message, it has to explicitly remove the message from the queue.
  • If the process fails to do so in a certain timeframe, because it crashed or isn’t done yet, the message will reappear on the queue.
  • And the message will be picked up by another process, executing the same logic again.

But hold on a minute! In your previous article you’re setting IsTransactional to true and now your telling me there are no real transactions, what’s up with that? Well, indeed, setting transactional to true means that we will play nicely along in this queue-peek-lock mechanism, setting it to false means we will immediately delete the message after receiving.

To avoid frequent occurrence of multi-processing you must ensure that your message handler completes before the timeout expires. By default the timeout is set to 30 seconds, but you can control it yourself using the MessageInvisibleTime property on the AzureQueueConfig configuration section (expressed in milliseconds).

But no matter what value you assign, multi-processing is always a potential problem on remote queueing systems, so make sure that your message handlers have been implemented in an idempotent way. In the future I hope to figure out a generic solution to this problem and add it to NServiceBus, but today I have no clue how to do it. I know Ayende divised the following solution using local transaction, but as azure table storage doesn’t support this kind of transactions it’s not really suitable in our case.

If you have any suggestion in this matter, I would, ofcourse, be happy to hear them…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: