In this case, the customer was running snapshot backups against their Exchange Server 2007 running on Windows Server 2008 (SP2). When trying to run a integrity check (eseutil /g) against the snapshot, eseutil was always reporting that the database was corrupt.
A quick check of servers found that the machine we were running the eseutil /g on was running Windows Server 2008 R2. While the ‘uneducated’ might think that Windows Server 2008 (with SP2) and Windows Server 2008 R2 would basically be the same OS version, the truth is much different; contrary to the naming (I still don’t know why someone made the “brilliant” decision in naming Windows Server 2008 R2), they are very different OS’es (Server 2008 is ‘Vista Server’ and 2008 R2 is ‘Windows 7 Server’).
Anyway, the issue here is that when we cross OS versions, the secondary indexes of the database have to be rebuilt to be compatible with the new OS. This is something that begins upon mounting the database on the new server with the higher level OS. Moving up in OS level (2003 to 2008 or R2, 2008 to R2) is supported, but moving back is not. What it looks like we're seeing here is that ESEUTIL is reporting corruption due to changing OS versions and the database needing the secondary indexes rebuilt.
To specifically test this, I built an Exchange 2007 SP3 server on Windows 2008 R2. I then took a database from one of my Exchange 2007 SP2/Windows Server 2008 SP2 servers and copied it to the SP3/R2 server. I attempted an eseutil /g and got back the -1206 error.
Then I mounted the database on the Exchange 2007 SP3/Win 2008 R2 server, dismounted and ran eseutil /g successfully.
Out of curiosity, I decided to test if I would see the same behavior if I took an Exchange 2007 database from a server running Windows Server 2003 (SP2) and ran eseutil /g against it on a Windows Server 2008 SP2 machine. I got the same -1206 error.
So, the conclusion here is that you cannot run eseutil /g without error on a different OS platform against a database, unless that database has been mounted on that OS to trigger the deletion/update of the secondary indexes first.
This information can be found in the Exchange 2007 Database Portability documentation: http://technet.microsoft.com/en-us/library/bb123954(EXCHG.80).aspx
Database Portability Across Windows Server Versions:
As with previous versions of Microsoft Exchange, an upgrade of the operating system for an Exchange server results in the updating of the value for OS Version in the database header. This update triggers the rebuilding of internal database indexes.
When using database portability to move a database from a Mailbox server running Windows Server 2003 to a Mailbox server running Windows Server 2008, the Extensible Storage Engine (ESE) detects the operating system upgrade and takes the following actions: • During the first database mount operation, all secondary indexes are discarded. A secondary index is used to provide a specific view of the mailbox data (for example, when messages in a mail folder are sorted using Outlook in Online mode). The database will not be mounted and available to clients until this initial operation is complete. The amount of time to complete the operation is largely dependent on the size of the database. The larger the database is, the longer the mount operation will take.
The customer then asked if there is a way to determine the OS version a database came from, and indeed we can. All we need to do is run eseutil /mh to dump the database header; in the output is an OS version line that shows the OS it was last mounted on.
Seriously ... I thought I was the only one who complained frequently, and constantly stresses to people, the importance of understanding the difference between Windows Server 2008 and Windows Server 2008 R2.