(or, How I Learned to Stop Worrying and Love /etc/inetd.conf)

1. Background

I registered my own domain name ten years ago and have been running my own email and web server for over half that time.  Until December of 2004, I was doing this on an OpenBSD machine running Sendmail.  It was solid and worked great.  About eighteen months ago, I joined the Exchange team, at which time, not running my email on Exchange started to feel like working at Honda but driving a Ford.  This wasn't a "company allegiance" thing; rather, working on a product that I didn't use myself felt disingenuous.

Getting Windows and Exchange up and running wasn't a problem.  Whereas I used to ssh to my BSD machine and read my mail via Pine, I now browse to my Windows machine and read my mail via OWA.  The problem was that, upon moving to Exchange, I suddenly had years' worth of messages that I cared about but that weren't accessible to me when reading my mail via OWA.

This blog describes the process of getting all that mail natively into my new Exchange mailbox.  By "natively," I just mean that the original timestamps, senders, and recipients were all the same once they landed in Exchange - as though they'd been there in the first place.  A lame hack that crossed my mind initially was to forward all my mail to myself, preserving none of those things.

For those that like this sort of detail, here's what I'm running:

    OpenBSD 3.5
    Sendmail 8.1340.2.189
    Windows Server 2003 SP1
    Exchange Server 2003 SP1

2. The Process

First, it would appear that I am not the first person to need to migrate old mailboxes to new ones - witness the Exchange migration wizard.  Invoking Start/All Programs/Microsoft Exchange/Deployment/Migration Wizard gave me the following window:

Left with little recourse, I clicked "Next" and got the following:

The highlighted option is the one that most closely matched my situation, though I didn't yet have an IMAP server running on my BSD machine.  So I left the wizard alone for a moment and visited OpenBSD's ports and packages page:

 http://www.openbsd.org/3.5_packages/i386.html

It's great - I get emacs, bash, pine, jove, etc, all from this page.  Searching the page (it's fairly long) for "imap" netted me several matches, and I decided on the IMAP4 server by the University of Washington.  Pulling the package down and installing it was painless, but the man page for imapd was extraordinarily terse.  It made reference to inetd but there was nothing in the way of explaining its command-line options, configuration settings, etc.  As a rule, OpenBSD's man pages are quite thorough.

Because I was feeling lazy, I just invoked imapd as root from the command-line to see what it would do - perhaps it would complain and issue a usage message from which I could bootstrap myself to greater understanding.  I learn about a lot of commands this way.  I also cut the tags off pillows.

Anyway, upon doing so, I got the following:

* OK [CAPABILITY IMAP4REV1 LOGIN-REFERRALS STARTTLS AUTH=LOGIN] howell.contoso.com IMAP4rev1 2003.339 at Mon, 7 Feb 2005 14:40:33 -0800 (PST)

Huh?

Apparently, imapd thought it was talking to an actual client.  Re-reading the man page reminded me of the whole inetd thing: UNIX daemons that expect to be invoked by inetd just open up stdin on the assumption that it's connected to a client via TCP (or UDP, or whatever).  So, adding an entry for imapd in /etc/inetd.conf and sending the SIGHUP signal to the inetd process had me up and running.  I verified this by pointing Outlook on my laptop at the IMAP server and verifying that I could pull down the messages that I'd last seen when reading my email via Pine.

Returning now to the Exchange migration wizard, I made it to the following step before encountering another speed bump:

What does a user list file look like?  Yes, I clicked the "Help" button.  A bit of searching microsoft.com turned up an example file.  Mine looked like this:

IMAP_Mailbox,SMTP_Address,IMAP_Password,IMAP_Server
sami,sami@contoso.com,password,howell.contoso.com

No, my password isn't really "password."

Clicking "Next" a few more times kicked off the migration process with a status window showing me the progress in terms of the number of files, errors, and so on.  This seemed like it was going to take a while, so I went off and did something else.  Upon my return about an hour later, things were still running.  This seemed weird.  I opened up the folder into which the migration files were being dumped and noticed that one of them was over 6 GB in size - I have nowhere near that much email, so I cancelled out of the migration and had a look at two of the smaller files (apparently, three files are created as part of migrating a mailbox).  One of them, 00000001.PRI, looked like so:

! Migration Type
MAILMESSAGE,"SMTP:sami@contoso.com",sami
! HeaderLine
Folder,Sender,To,Cc,Bcc,Subject,Send-Date,Received-Date,Priority,Unread...
"mail\sent-mail",,,,,,,20020901233714,,TRUE,FALSE,#00000001.SEC(66),0,0,
"mail\sent-mail",,,,,,,20020906105428,,TRUE,FALSE,#00000001.SEC(381),0,0,

Oh... it looks like the migration just recurses through the mailbox of the user whose mail is being migrated, and imapd was just handing back every single file and folder in my UNIX home directory as a "mail folder."  As it happens, I've got around 6.5 GB of photography and music files in my home directory, and those were all being brought over.  Not what I intended, though apparently the migration wizard can handle absolutely massive mailboxes (know anyone with 6GB of mail?) without choking.

Since the migration was cancelled, my Exchange mailbox was still untouched, which was a relief.  Back on the BSD machine, I figured I'd "hide" all my files except for the mail files, run the migration wizard, and unhide them later.  Here's what I did:

[howell /home/sami 525] mkdir bak
[howell /home/sami 526] mv * bak
[howell /home/sami 527] cd bak
[howell /home/sami 528] mv mail ..
[howell /home/sami 529] cd ..
[howell /home/sami 530] chmod 000 bak

If you're paying attention (and you know UNIX), you'll see that I omitted a few things, such as the moving of the dot files in my home directory, and the error that I must have gotten when attempting to move the bak directory into itself (yes, I could have moved everything to /tmp, but moves within a single filesystem are basically just pointer updates rather than actual copies of data - recall the 6.5 GB of photos and music).  That aside, I was golden as long as imapd honored the permissions that I'd set when doing its traversal.  Re-running the migration proved that it had because the migration ran in less than a minute this time!  Phew.

Refreshing my folder list in OWA showed a new top-level folder called "mail", whose name came from the "mail" subdirectory in my UNIX home directory.  Inside that folder were sub-folders named after each file that Pine had created to store my mail.  All that remained was a bit of re-organization to move these migrated messages into folders that I'd created since moving from Pine to OWA.

3. Denouement

In case you were wondering, I still do my web serving off the OpenBSD machine.  Hey, I like Apache, I like Perl, and I like CGI scripts.  If I ever move to the IIS team, perhaps I'll rethink this, too.

- Sami Khoury