Understand why messages get “stuck in the Outbox folder” in an Exchange environment.

This is a taste to whet your appetite. For the complete story, open the attached .PDF file.


(Today's post is courtesy of Scott Bradley, Principal Escalation Engineer, Office Technical Support.)


A trip back in time—the evolution of Outlook


A thorough understanding of mail submission/delivery starts with a trip back in time. Years ago, before Outlook was born, Microsoft defined some standards for messaging applications. These standards are commonly referred to as MAPI (Messaging Application Programming Interface), and detailed documentation can still be found on MSDN. At the root of the MAPI world is this idea of three main messaging components:

  1. A place to actually STORE the messages—the MAPI MESSAGE STORE
  2. A method to SEND the messages—the MAPI TRANSPORT
  3. A place to store and look up ADDRESSES—the MAPI ADDRESS BOOK


Most people are familiar with the common stores and address books in the Exchange/Outlook world. We have the mailbox and Global Address List (GAL) from Exchange, and things like the PST/OST file and Offline Address Book or Contacts Folder (which is abstracted into an address book by the Outlook Address Book service) from the Outlook side. But this middle part of MAPI, the transport layer, is less understood.


In non-Exchange scenarios, the transports are much easier to understand. Consider the POP/SMTP setup. You use the POP protocol to receive mail, and SMTP to send mail. The protocols are easy to read and understand. You can look at a network trace and see the commands and the mail “going over the wire,” etc. In the Exchange scenario, things are more complicated.


Along with this notion of the MAPI TRANSPORT being responsible for actually sending mail, MAPI includes the idea of a “spooler” which is something that manages the transports. It loads them up, checks for mail that needs to be sent and actually calls into the transport’s code to send the mail. You can think of it as the “driver” of mail delivery. Until Outlook 2002, the spooler was implemented in a completely separate executable. So when you ran Outlook (or any other MAPI application), you would also see a process named MAPISP32.EXE in your task list. The spooler ran independently of Outlook. When it came time to send mail, MAPISP32.EXE “woke up” and did the work of driving the mail.


In Outlook 2002 the design changed, and the spooler code was moved into the Outlook executable itself. This design still exists today and has some interesting implications. Understanding these implications means you really have to embrace the fact that Outlook IS the spooler. Anything that you expected to happen with the spooler, now only happens when Outlook is running.


It’s easy to see the result of this design when you send mail from another application when Outlook is not running. Consider this scenario:

  1. Outlook is installed but not running on your computer.
  2. On the desktop, you right-click a file and choose “Send to Mail Recipient.”
  3. Outlook comes to life and gives you a message to work on.
  4. You type a recipient name and a message, and hit Send.
  5. Later you shut down your computer and go home.
  6. The next morning, the recipient is mad because they never got the mail.


What happens in this case is that immediately AFTER you hit send in step 4, Outlook is done, so it closes. Now Outlook is not running, and therefore no spooler is running, and so nothing is going to send your mail. The next time you launch Outlook, you will see the mail in the Outbox. Assuming Outlook stays open long enough for a spooling cycle to happen (default is 15 minutes), the mail will be sent successfully.


One easy way to conceptualize the spooler in Outlook is to think of it as “everything in the Send Receive Groups” dialog. This dialog roughly maps to the things the Spooler code does and how Outlook manages all the options.


  
As an aside, look closely at the second screenshot. This is from a standard cached mode configuration of Outlook. You will notice that almost all the options are gray and unavailable. This is because the Exchange account is NOT included in this group in a standard cached mode configuration. Note the red X over the account on the left, and the empty checkbox at the top. In cached mode, you do NOT typically want the send/receive group settings to apply. Cached mode has its own logic for synchronizing and sending mail, and it is notification based, not scheduled like the send/receive groups.


Finally, consider the impact of the cached mode feature that came along for Outlook 2003. Cached mode is many things, but one specific part of the cached mode design is that mail delivery uses this spooler process in a more intricate way, and uses it for ALL cached mode sends. Again, it works independent of the settings in the Send/Receive Groups, and it has switches and configuration choices that add complexity.


So there are four key takeaways to start our understanding of Outbox problems:

  1. MAPI Transports are the way that Outlook “sends mail”
  2. There is this notion of a spooler that is the key to managing and driving the sending mail cycle
  3. Outlook.EXE *IS* the spooler ever since Outlook 2002
  4. The cache mode feature drastically changed the importance of the spooler

 

Mail delivery in Exchange


Up until now, I have described the general MAPI ideas around mail delivery and how Outlook implements these options. Now we need to understand some specifics in the Exchange Server configurations. Again, some history is needed to help understand how we got to where we are now.


In early versions of Outlook/Exchange, users almost always used Outlook in the “online” mode of operation. You interacted with Mail and Addresses directly from the Exchange Mailbox. This is in contrast to today’s world where most people use cached mode. In cached mode, you work with data and addresses from a LOCAL data store (your OST file), and then synchronize your data to the mailbox. Note that SENDING mail is not the same as synchronizing. The Outbox folder is never part of synchronization. Mail submission and delivery is different from synchronizing.


There is a fundamental difference in mail submission/delivery between online and cached mode. Understanding this difference is one big key in your ability to grasp the entire Outbox story. For this blog entry, I am going to define two different methods to deliver mail:

  1. Transport Send (this is the spooler method that I described in detail above)
  2. Store Send (this is the second method that ONLY is used in the Exchange Online scenario)


Since we know that in cached mode you work from your OST and not online, you can see that all cached mode mail delivery must happen using the Transport Send method. If you are running in the old online profile (which is common even today on terminal server configurations), then mail delivery is typically done using the Store Send method.


So why do we care which method is used? Mainly because a Store Send does NOT require any of the spooler process to happen. When the Store Send method is used, the message is delivered by the server with no further client interaction. In the “send to mail recipient” scenario that I described in the previous section, if the Outlook profile is configured for online mode, that mail *IS* sent successfully. Since no spooler is needed to deliver the mail, it is delivered immediately by Exchange, whether Outlook is running or not. There are other differences “under the hood” for each method. Appendix C in the attached .PDF complete version of this blog has a table that summarizes the differences.

 

How Outlook chooses which delivery method to use


A good way to conceptually think about the Outlook decision process is to think in terms of the Outbox folder itself. When a message is created in the Outbox folder of the OST/PST, Outlook and MAPI hook up all the plumbing so that mail submission happens through the Transport Send method. When a message is created in the Outbox folder of the online mailbox, Outlook and MAPI hook up the plumbing so that mail submission happens through the Store Send method. So using this concept, if you are in cached mode you should expect a Transport Send, and if you are in online mode, you should expect a Store Send.


Additionally, even if you are in ONLINE mode and would typically be using the Store Send method, there are options and configurations that will prevent the use of the Store Send. The most common example is the presence of an additional transport. So for example, if you have an online configuration and you ADD an SMTP/POP account, you are no longer eligible for delivery via the Store Send method. Likewise if you add a meeting service transport like the LiveMeeting service, you must now send mail using the Transport Send method. Any additional transport will “turn off” the ability to do Store Send with Exchange. Finally there are a couple of Outlook configuration options that will “turn off” the Store Send in online mode. The primary one is the option to save replies with their original mail. So to summarize, you can only take advantage of the Store Send method if:

  1. You have Exchange Server.
  2. You are configured for online mode and not cached mode.
  3. You have no other transports in the profile.
  4. You are not using the “save replies with original” option.


The formal term for the “save replies with original” is this setting in the options dialog:

 

The NEEDS_SPOOLER flag


This option gives a good example of an important “extra” trigger in the mail delivery process. Even if you are using the Store Send method, where the spooler (Outlook) is not required, there are scenarios where the mail will require the spooler. The “save replies with original” feature requires some processing by the spooler. So even in a Store Send scenario (online mode, no extra transports, etc.), Outlook will add a special flag to the message during submission that tells Exchange not to deliver it immediately. This flag is called the NEEDS_SPOOLER flag.


The process looks like this from a non-technical point of view:

  1. Outlook’s logic runs through some tests and determines that this message will need the spooler for some reason.
  2. When Exchange accepts the message for submission, it looks for the “this message NEEDS the spooler” trigger.
  3. If there is no trigger, then Exchange sends the mail itself and you have a Store Send.
  4. If the trigger is there that says “this message NEEDS the spooler,” then Exchange does NOT deliver the message and waits on another command from the spooler before sending.
  5. When the spooler cycle happens, it sends the message up to Exchange to “Send the message” and you have a Transport Send.


So all the criteria are tested for each method and in the end, each message either gets the trigger for “needs the spooler” added or not.


Notice that if Outlook calculates that a Transport Send is needed, there is some amount of time between message submission and message delivery. In a Store Send, there is only the one submission, but in the Transport Send, you have the message submission, and then at some later time, the command to do the Transport Send coming from Outlook itself. There is a choice in the Outlook options screen called “Send Immediately when connected” and the option is on by default. When this option is on, Outlook will kick of a spooler cycle and do a Transport Send immediately after sending the message if a spooler cycle is needed. If the option to “Send Immediately when connected” is turned off, then the message will only be sent according to the defined send/receive settings. Since the send/receive settings, by default, do not include the Exchange Account for cached mode, it can be pretty easy to have a send/receive definition that you do not expect. So it’s important to examine the send/receive settings and make sure the expected definitions exist for how often you want to send mail. Below is the screenshot showing the “Send Immediately” option.


 
As a summary of this section, here is the way Outlook thinks about how a message gets submitted and delivered:

  1. Is this new message I am sending based in an online Outbox folder or an OST/PST folder? If it’s online, route the submission through the Exchange “Store Send code.” If it is OST/PST, route the submission though the “Transport Send” code.
  2. After submitting the message, do a bunch of checks to see if either this is a straight Transport Send, or one of the special case Store Sends that “need the spooler.” If either of these is true, then kick off a spooler cycle to do the Transport Send.
  3. If this was an OST/PST based send, then do the work to move the item from the Outbox folder to the Sent Items folder, and send any NDRs that have occurred.


So we’ve unlocked some of the Outlook Outbox mysteries, but there are more! Open the complete paper attached to this blog. You’ll learn all about:

  • Submission state
  • The Outbox
  • Synchronization
  • The spooler cycle
  • Address types
  • Non-delivery reports
  • Troubleshooting issues, including
    • Transport logging
    • Synchronization issues folder messages
    • Registry settings
    • MFCMAPI