WS-RM Flow Control in Indigo
It is natural to assume all reliable messaging systems will implement some sort of congestion and flow control algorithms. This is bound to happen because we are driven to develop smart and efficient algorithms that will minimize the load on the computers and networks that support them. Indigo's WS-Reliable Messaging implementation is no different – it puts to use advanced congestion avoidance algorithms that have been tailored to the specific intricacies of SOAP messaging, the WS-RM protocol itself, and the kinds of networks WS-RM is expected to be used on. I will discuss what Indigo does about congestion control in the next installment of my blog; today, we will concentrate on WS-RM's flow control mechanisms.
You may have noticed a custom XML element appearing in all acknowledgements Indigo endpoints send: BufferRemaining. A concrete instance, inside a SequenceAcknowledgement, will look like this:
<r:SequenceAcknowledgement>
<r:Identifier>urn:uuid:7c96e44a-2158-44d5-aee3-f0daeec853e0</r:Identifier>
<r:AcknowledgementRangeUpper="1" Lower="1" />
<netrm:BufferRemaining xmlns:netrm="http://schemas.microsoft.com/net/2004/02/RM">31</netrm:BufferRemaining>
</r:SequenceAcknowledgement>
If it existed, the normative outline for that element would be: <netrm:BufferRemaining ...> xs:unsignedLong </netrm:BufferRemaining>. By the way, this is currently the only element you’ll see in the http://schemas.microsoft.com/net/2004/02/RM namespace.
Now that we’ve covered its syntax, let’s cover its semantics. Its purpose won’t come as a big surprise: to inform the sender of the current status of the receiver’s buffer for incoming messages. More specifically, the number represents the number of messages the receiver would be able to accept and buffer from this point on before beginning to refuse them. The number will go down every time the receiving RM channel buffers a new message and go up every time a receiving application accepts a new message from that channel. The maximum size of the receiving channel’s buffer becomes the maximum value a BufferRemaining element can ever hold – in Indigo, this size is controlled by the BufferedMessagesQuota property on any RM enabled binding. By default, the value is 32.
Whenever an Indigo sender receives an acknowledgement, it updates its perceived size of the free space available on the receiver. The sender must do this because the actual value it uses is continuously changed as the sender sends out new messages. In other words, what the sender actually believes is the remaining buffer size is calculated by subtracting the number of outstanding messages (i.e. messages that have not yet been acknowledged) from the latest BufferRemaining value it received. For example, by default, if the sender sent 32 messages before receiving an acknowledgement with a BufferRemaining of 31 and which acknowledged the first message only, its perceived remaining buffer size would be 0.
An Indigo sender uses its perceived remaining buffer size in two important ways. It will begin piggy-backing AckRequested elements on all new messages it sends if the value drops below a certain point (currently 2) – it will do this to increase the chances of receiving an acknowledgement with an updated BufferRemaining that will hopefully indicate the real remaining buffer size on the receiver isn’t so low. More importantly and more obviously, it will stop sending any new messages and allow the server to catch up if the value ever drops down to 0. The only event that will resume message transfer is an acknowledgement from the receiver that free space became available. An Indigo receiver will send such a sole-purpose acknowledgement immediately after it detects it. You may be asking what happens in the case an asynchronous transport is used and the receiver can’t talk to the sender unless the sender talks to it first. We don't want to wait for an AckRequested keep-alive (which would only be sent, as another blog post will explain, after half the inactivity timeout), so what an Indigo client will do is begin periodically polling the receiver with AckRequested requests until it gets back a response that indicates new free space. Note that flow control over asynchronous transports wasn't implemented in Beta 1.
Finally, the BufferRemaining value always represents the actual free space regardless of the ordering of the session. This may be confusing at first. If the receiver is instructed to deliver messages in order, it must always ensure there is enough space in the buffer for all messages with message numbers higher than the last message received by the application and lower than that number plus the quota. For example, if the quota is 32 and no messages have been received so far, only messages 1 through 32 would be buffered even though the buffer is completely empty and BufferRemaining was 32. Any other message will be thrown away.
