(or, The Design of Exchange Direct Push in Exchange 2003 SP2)
I bought my first cell phone in the summer of 2000 – the venerable Nokia 5160. With the attendant giddy excitement of a new consumer electronics purchase, I started adding names and numbers to the addressbook on the phone. This bubble was to burst when I would arrive at work to realize that the contacts and appointments I’d spent the last few years entering into Outlook were isolated from those on the phone. There may have been custom solutions to keeping the two synchronized, but a cursory search on Nokia’s site did not yield anything obvious – certainly, nothing out-of-the-box existed at that time for my setup.
Fine, I thought. For now, I’ll resign myself to manually entering each contact twice – once on the phone, and once in Outlook. But this sucks.
And so it was until the Nokia 8390 came out. I bought it on literally the first day that Seattle’s AT&T Wireless store stocked them, as this was the first phone offered by Nokia that met my specifications:
Note that between the 5160 and the 8390, I also owned the 8260 (yes, the Charlie’s Angels phone). Consider what this means: Upon buying the 8260, I had to manually copy every contact from the 5160 to the 8260 – if the store personnel had the capacity to use custom tools to do this for me, they didn’t offer. And this process had to be repeated when I moved from the 8260 to the 8390. Finally, all the while, I ran the risk of losing my current phone and, with it, all of my carefully-entered contacts. Best case, this meant an incredibly laborious, error-prone, eye-crossing two hours spent manually entering contacts into my replacement phone – and only if I happened to be diligent about doubly entering every contact into both Outlook and the phone. Worst case, this meant some lost contacts and the all-too-familiar emails whose contents read something like, "I’ve lost my phone and your phone numbers; please send me your contact information!"
If the double-entry of contacts in Outlook and the phone sucked, this really sucked.
With the 8390 in hand, I was ready to trounce several of these issues:
I installed the synchronization software onto my laptop and connected the 8390 using infrared. Problems surfaced immediately upon completing the first synchronization, however:
Ok; so, annoying, but not the end of the world. Some of this can be attributed to ambiguities in mapping from one set of schema for contact and calendar items to another, to limited storage on the device, and perhaps to early versions of the software (though I was using version 4). I lived with this for about a year, burned through two 8390’s, upgraded to the Nokia 7210, and lived with that and a similar synchronization experience for about another year.
During that time, things were manageable: the phone upgrades I mentioned were fairly painless because my Exchange account contained the authoritative copies of the data, and, with each new phone, I just pulled down all the data by way of Outlook at the first synchronization. However, it was still less than ideal: upon modifying a contact or an appointment on my calendar, I had to remember to align the infrared ports of my phone and laptop (I tried repeatedly and unsuccessfully to find a USB cable for connecting the two), kick off the synchronization software, and wait for it to complete. Making this a part of my daily routine was just tedious.
Enter the Motorola MPx200. This was the first phone offered by my service provider that ran a Windows Mobile operating system and was of an appropriately small form factor. The significance of the phone running a Windows Mobile operating system is that the phone would be running ActiveSync, which would provide high-fidelity synchronization of the email, calendar appointments, and contacts in my Exchange mailbox. This would be supported both over-the-air with a GPRS connection and via USB/infrared using the desktop ActiveSync software that was provided along with the phone.
A further nicety of the MPx200 was the cradle and the single power/data USB connector it used – this meant that I could cradle the phone on my desk upon arriving to work, let it charge and sync all day, and pluck it from the cradle upon heading home, fully charged and synchronized.
The Design of AUTD
So, things are looking pretty good now (our story began in the summer of 2000; it is now the fall of 2003). What’s the problem, then? Well, the device synchronizes itself on a schedule, the most frequent setting of which is every five minutes. I’ve always interpreted the setting "How often would you like to sync?" as "By how much would you like to be out of date?" This meant that I may well pluck the device from its cradle before the next scheduled sync has occurred and miss some updates. Further, scheduled syncs over the air are fairly costly: most folders don’t contain changes. Finally, these unnecessary syncs cost power and adversely affect the lifetime of the battery of the device.
Yes, we offered an always up-to-date (AUTD) solution based on text messaging at that time, but I wasn’t happy with what was required in terms of provisioning and the server-side enforcement of latency so as to mitigate the impact of AUTDv1 on server performance.
What to do.
Around this time, we had begun looking into what it would take to offer an up-to-date mobile email solution ("AUTD," from here on) that competed with the likes of RIM, Good, et al. I liked the up-to-date nature of their solutions but had not personally adopted them for reasons of device choice (again with the form factor), setup costs (in terms of money, deployment overhead, and operational overhead), or both. Being on the Exchange team, we’ve always got two sets of customers: the administrative staff and end users, and we wanted to build a solution that worked well for both. By enumerating our requirements and constraints, we essentially painted ourselves into a corner (happily, this corner contained the solution):
Within this definition of the problem, we came up with the following solution:
I’ve omitted some details here, but that is what is going on under the covers when you check the "Enable up-to-date notifications via HTTP" checkbox in Exchange System Manager in Exchange 2003 SP2, and it has the benefit of working on any mobile operator network that supports internet connectivity. Since the hopes of increased revenues of most mobile operators appear to be pinned on the possibility of selling users on data-enabled applications, this seemed like a safe enough bet.
Further, by using HTTP, we do not require enterprises to open any inbound ports beyond what they’ve already had to open in order to support Outlook Web Access (OWA), Outlook’s RPC-over-HTTP feature, and ActiveSync itself. Finally, the client-initiated nature of HTTP makes the device ultimately responsible for connectivity with Exchange, so upon receiving the request for change notifications from the device, Exchange will return a response immediately if any changes have occurred since the last synchronization. This is how we prevent "dropped" notifications. If the device ever drifts out of coverage, it will enter a re-try loop and connect as soon as it is able. The network resilience logic of the device can also be triggered on the timeout limit having elapsed before a response from the server is received.
So there we have it: an up-to-date mobile email solution that is friendly for administrators and users alike. Changes trickle into the phone in the same way that they do into Outlook on the desktop. In fact, updates appear on the phone before they do in Outlook and OWA!
Now then, if you’ve been paying attention, you’ve probably noticed that AUTD requires a persistent data connection twixt the device and Exchange, and you’ve got a few issues with this:
What the previous three points add up to is that AUTD is actually better for mobile operator networks and device battery life than the solution based on scheduled sync that is used by devices that mobile operators sell today. We’ve had a bit of difficulty in getting this point across to some mobile operators.
As you might guess, we’ve been running early versions of this in Exchange for a few months now. One of the more satisfying testaments to the utility of our AUTD solution is watching upper management bump into each other in the halls as they consult their devices for the email that just arrived or for the location of the meeting for which they’re already ten minutes late.
To let out a little secret, I’m not actually all that interested in having up-to-date email on my phone, though that aspect of it is a big favorite for our upper-management types around here. For me, having updates that I make to my calendar and contacts “just appear” on the phone without any special, conscious action on my part was the motivating idea behind all of this.
 Ok, consider what this means besides the apparent fact that I like Nokia phones. AT&T Wireless may have carried PDAs running the PocketPC operating system around that time, but, on social grounds, I refuse to carry around one of those things. The ActiveSync protocol is proprietary but Microsoft has begun licensing to third party vendors like Motorola, Nokia, PalmOne, and Symbian. Given that, the choice of devices that allow for high-fidelity synchronization with Exchange is no longer limited to those running Windows Mobile operating systems. Motorola, if you’re listening, this is one of the omissions from the MPx220 that is keeping me from buying one. Truth be told, I am actively disinterested in having email from work constantly appearing on my phone, but my boss is probably reading this.