Evan Dodds - Microsoft Exchange Server Blog : Cross-site Moves

Permissions required for mailbox moves

evand — Tue, 29 Aug 2006 23:02:02 GMT

Ross just posted a detailed view of the permissions required to do both cross-site and intra-site mailbox moves over at the EHLO blog: http://msexchangeteam.com/archive/2006/08/29/428781.aspx.

Great post if you’re looking for more info on what permissions your admin account will require to do the moves. If you don’t have all of the required permissions, for instance, you may get the “Unable to find a responsive Exchange 5.5 server in the specified site. ID NO c1034a34” error I detailed in this previous blog post (when your admin console cannot successfully connect to the target SRS directory).

Can't run Exdeploy after installing W2k3 SP1?

evand — Mon, 04 Jul 2005 19:56:00 GMT

Here’s an interesting problem: After you apply Windows 2003 SP1, the new security settings prevent running HTA files from a network share. So this means if you download the Exdeploy pack and dump it into a network share, it won’t work properly on a W2k3 SP1 server.

The workaround, of course, is to put the files locally on the W2k3 SP1 server.

Seems to me that people most affected by this should be those who are doing cross-site mailbox moves, since anyone doing a regular in-place upgrade from Exchange 2000–>2003 wouldn’t yet be running the server on Windows 2003 SP1.

See KB.897340 for more info.

Cross-site moves: The importance of proper ADC Connection Agreements

evand — Tue, 24 May 2005 19:42:00 GMT

It’s now been about a year since Exchange 2003 SP1 released, and cross-site, mixed-mode mailbox moves were introduced. In that time, I think folks have really started to get the hang of how it works and what needs to be in place to ensure successful moves.

That said, there’s one topic that I’ve seen popping up more and more frequently over the past few months (perhaps it’s just because some of the other common questions are starting to recede in volume as more and more documentation hits the web).

“Why won’t my stub objects go away?” (or its close cousin — “why do I keep getting NDRs long after I moved the mailbox?”)

Ok, so you might say “hey Evan, didn’t you already cover that in this post”

Yup. That post covers the under-the-hood details of the move, and will probably help you understand where the process is breaking down. Have a look at it if you haven’t yet.

But, I’ve had it brought to my attention that there’s one area of this post that I might not have emphasized enough: You Have To Have The Proper Connection Agreements In Place!

Almost every step in the cross-site, mixed-mode mailbox move process relys on the proper ADC connection agreements being there. The Exchange Deployment tools (download here) guide you through the whole process, and actually have you run the “ADCTools” to create and confirm the proper CAs several times during the process.

This is very important. If you don’t have the proper ADC Connection Agreements in place, you will run into very weird “blocking” problems where objects don’t get created/deleted/updated, etc.

One specific case I’ve had brought to my attention is the scenario described in KB.888032. This is the case where you have a “pure” Exchange 2003 site (ie - a site with only E2k3 in it, no 5.5… no SRS). In this scenario, another site has an SRS that “owns” this pure E2k3 site. Various updates for the “5.5 view” of this pure E2k3 site have to be handled by this particular “owning” SRS. But since it’s a “pure E2k3” site, it’s really quite easy to forget that even this site needs to have a CA too (and that it needs to point to this SRS in another site.. an altogether confusing situation for most people!)

So, in summary, if you’re running into problems with the “transition” part of your cross-site, mixed-mode mailbox moves, I suggest you start with the following:
1) Have a look through my original detailed process post and see where you’re getting stuck.
2) Make sure you’ve run the ADCTools and/or confirmed your CAs are correct
3) Have a look through KB.888032 to make sure it’s not the issue you’re hitting

Funky mail routing after cross-site mailbox moves

evand — Thu, 17 Mar 2005 16:07:00 GMT

When you move mailboxes across mixed-mode site boundaries with the Exchange 2003 SP1 feature, there is a period of time where mail will have trouble routing properly to these moved mailboxes. There are a couple of ways you may see this presented, as I’ll discuss in this post.

The most common case is the Non Delivery Report (NDR) scenario. That is to say that after you move a user’s mailbox across administrative group boundaries in mixed mode, mail sent to this user may be returned non deliverable. This is discussed in KB.841659. The short explanation is that:

Mailbox is moved from Site1 to Site2 (so the data is now in Site2, not Site1)
5.5 directory in Site1 is updated to make the original mailbox a “stub” object which cannot receive email
AD is updated to reflect that the object exists in Site2
5.5 directory in Site2 is not updated until ADC replicates from AD->Site2 (so there is not yet a directory object for the mailbox in the Site2 5.5–world)

While we’re in this transitioning phase, if someone on a 5.5 server tries to send to the moved mailbox the 5.5 directory will be referenced. The 5.5 directory is not completely updated at this point, so the message will eventually hit a 5.5 server that doesn’t know how to deliver mail to this mailbox object (for instance, the 5.5 directory in Site2 hasn’t yet had the ADC create the new object, so it has no knowledge of this mailbox from a directory perspective).

This happens right up until the ADC replicates, directory replication completes, and the stub object is removed – see this post for more details on this process.

There’s also another scenario I’ve seen that has slightly different results. The scenario is that you have no RPC connectivity between the server that is the source of the move and the server that is the target of the mailbox move. A common way this may be the case is for the two sites to be connected by X400 connectors, and not share a common service account.

During the mailbox move (as in all mailbox moves, see my EHLO post for more info), the PR_IN_TRANSIT property is set on the mailbox to prevent new messages delivering to the mailbox while it is moving. In most cases, the source of the mailbox move is a 5.5 server, so when the move completes, any messages that had queued up during the move are released back into the MTA for delivery.

The MTA isn’t really smart about cross-site mailbox moves, so it makes the assumption that (of course) the target of the mailbox move has to be in the same site, which means that (of course) we will have RPC connectivity to the server where the mailbox now resides.

Except in the scenario I’ve described above. With X400 connectors and no shared service-account, there’s no guarantee we’ll be able to make an RPC connection to the destination server. Enter the problem: The MTA short-cuts the mail routing, and just drops these queued messages into an RPC queue, destined for the servername of the destination server. The typical symptom is that you’ll see a mail queue on the source 5.5 server destined for the (netbios) name of the target server with queued items in it, even though new mail destined for this target server is flowing just fine across the appropriate connector. You may also see events logged from the MTA indicating a failure to connect to the server in the other site.

This doesn’t affect all environments, as RPC connections are often possible between sites. If you have a site connector between the sites, this behavior is totally transparent and the mail will route just fine. That’s the suggested workaround – just create (even temporarily) a site connector between the two sites so that these mail items can drain out of the queue and deliver.

Couple of Interesting KBs

evand — Wed, 16 Mar 2005 20:14:00 GMT

Here are a couple of interesting KBs that have come out or been updated recently:

KB.884863 (http://support.microsoft.com/kb/884863) – “Update to Exchange 2000 Server adds application event logging for deletion of public folders”

An additional feature that is related to public folder diagnostics logging is now available for Microsoft Exchange 2000 Server. When you use this new functionality, an event is generated in the Event Viewer Application log when a user deletes an Exchange 2000 public folder. If the Public Folders General category is set to Medium logging or higher, an event that similar to the following will be logged when a user deletes a public folder in Microsoft Exchange 2000 Server:

Event Type: Information
Event Source: MSExchangeIS Public Store
Event Category: General
Event ID: 9682
Description: Folder /Name_of_Public_Folder with folder ID 9-9999B was deleted by /o=Exchange_Organization_Name/ou=Exchange_Site_Name/cn=Recipients_Container/cn=User, user account Domain\NTAccount_Name.

Note, you can also get this hotfix for Exchange 2003 Post-SP1 by referencing KB.891968 with PSS.

KB.895847 (http://support.microsoft.com/kb/895847) – “Multi-site data replication support for Exchange 2003 and Exchange 2000”

You can use replication technology to help provide high availability for Microsoft Exchange Server 2003 or Microsoft Exchange 2000 Server data. Exchange 2003 and Exchange 2000 replication solutions are provided by third-party program vendors. This article describes our support policies for Exchange when Exchange is used together with a third-party replication solution.

(Translation: this is the definitive “Exchange on replicated storage” KB article. It covers synchronous replication, asynchronous replication, software replication, Geo-clusters, you name it!)

KB.894094 (http://support.microsoft.com/kb/894094) – “You can take the MTA resource offline, but you cannot bring the MTA resource back online after you restart a Windows-based cluster”

On Exchange 2003 clusters – particularly clusters with lots of MDB stores – you might find that if you cycle the MTA resource in CluAdmin, the resource fails to come back online. This is because in some scenarios, the DB000001.DAT file may be corrupted by this action. Please see the KB for a bit more detail, and get the hotfix if this applies to your large cluster.

KB.175481 (http://support.microsoft.com/kb/175481) – “Exchange single-instance storage and its effect on stores when moving mailboxes”

This KB article about single-instance storage behavior during mailbox moves (ie – that we keep it if you use the built-in move process) has been updated to include the cross-site mixed-mode Exchange 2003 SP1 feature. The article clarifies that these cross-site moves also retain SIS, unlike the old “Exmerge” method of moving mailboxes cross-site.

Do my cross-site mailbox moves have to be from a 5.5 server?

evand — Tue, 01 Mar 2005 01:37:00 GMT

Common question I’ve been getting lately is “does the source of a cross-site move with the Exchange 2003 SP1 functionality have to be a 5.5 server?”.

The answer is no. You can move mailboxes cross-site if the mailbox is currently on 5.5, 2000, or 2003 servers… the biggest prerequisite in this area is that the target of the move has to be an Exchange 2003 SP1 server.

In at least one of the cases where I was asked this question, the customer was having problems with the move, and therefore assumed that it was because the source was not a 5.5 server. In that case, they were receiving an event ID 9164 and event ID 1008 during the move:

Event ID: 9164
Description: Failed to open object <mailbox> on server <servername>. Error: There is no such object on the server.

Event ID: 1008:
Description: Unable to move mailbox <mailbox>. There is no such object on the server.

You will get these errors if the move mailbox process is unable to find the 5.5 directory object that is associated with the source mailbox. You’ll probably never have this problem with a source mailbox on a 5.5 server, as the mailbox would be totally broken in that case. Where you will see this is when you have a mailbox on an Exchange 2000/2003 server and the ADC has not replicated the mailbox information from the AD into the 5.5 directory. When the move mailbox process tries to update the object in the 5.5 directory (as it does during cross-site moves, see here), it cannot find this 5.5 directory object and logs the error.

Even if you’re moving cross-site from a pure Exchange 2000/2003 site, there is still (somewhere) an SRS which owns this site, and on that SRS should be a directory representation of each and every Exchange 2000/2003 mailbox in the site. This is how 5.5 gets to know about the Exchange 2000/2003 recipients in that pure site.

The most common causes I’ve seen for these sort of failures are:
1) Not waiting long enough — ADC hasn’t had a chance to replicate yet, but it will in time or if you “replicate now”
2) ADC misconfigured — no matter how long you wait, if the ADC isn’t configured right, you may not see these objects replicate (Suggest you run ADCTools)

Hotfix for scripted cross-site mixed-mode moves!

evand — Fri, 04 Feb 2005 18:46:00 GMT

Thanks to Dave Howe for pointing out this newly released hotfix. With the release of KB.883652, it’s now possible to script cross-site mixed-mode moves based on the Exchange 2003 SP1 feature. Before applying this hotfix, if you are doing scripted moves with the MoveMailbox method of the IMailboxStore interface and you select a destination in a different site you will get an error like:

This mailbox cannot be moved to a different Exchange Server 2003 administrative group while in a mixed Exchange 5.5/Exchange Server 2003 Organization.
Error code: -1056749031

With the hotfix in place, a new parameter (FCrossAG) is exposed that allows you to specify that the move should be cross-site. Also included with this hotfix is exposure of a parameter that allows you to control the Skip Bad Items feature of Exchange 2003 mailbox moves (IMaxBadItemsToSkip).

Please see KB.883652 for more detail on how to use these changes and download the hotfix here.

More DS/IS prerequisite validation info for cross-site, Exchange mixed mode mailbox moves

evand — Tue, 25 Jan 2005 18:24:00 GMT

Back in July of last year, I posted about how the DS/IS hotfix prereq check works and why you need to apply it. Since that time, I’ve run into one other item I wanted to talk about in the process that I didn’t provide enough detail on in the original post.

Here’s a recap of the “check for the DS/IS hotfix” prereq steps from the earlier blog post:

How we check for the DS/IS hotfix during cross-site moves:
When moving mailboxes or DLs cross site, the tools check to see if the hotfix has been applied to ALL 5.5 servers that have a public folder store. The move mailbox GUI is even nice enough to provide a list of servers where it doesn't believe the hotfix has been applied. So what do you do when you are POSITIVE that you've applied the hotfix to the “EXMAIL4” server, but the GUI says it's not? Why does this happen?

The process to check for the DS/IS hotfix is roughly as follows:

Figure out what our target site is -- what site are we moving the mailbox (or DL) to? For the sake of example, let's say we're moving the mailbox from “Site1“ to “Site2“.
Look through the list of Config_CAs to see which one “owns“ this naming context. One of the Config_CAs will have this site listed in msExchServer2ExportContainers, and this is the Config_CA (and therefore the SRS) that owns this 5.5 site.
Open a connection to this SRS. If the SRS is offline or unavailable, we're done here and the prereq fails.
Query the SRS for a list of all 5.5 servers that have a public folder store defined.
Scan the list of 5.5 servers for the heuristics attribute on each server object. If the server object in this SRS directory doesn't have the proper heuristics attribute (to indicate that the hotfix has been applied), we fail the prereq.

And that's it; if we DO have the proper heuristics attribute stamped on all of the servers in our query, we pass the prereq test and move on.

Notice in Step #3, “Open a connection to this SRS” is sort of undefined. After Step #2, we’ve determined which SRS owns the site which is the target of the move. But if we try to open a connection to this SRS and fail to connect it for some reason, what happens?

Well, a very common error code to see in this case is: “Unable to find a responsive Exchange 5.5 server in the specified site. ID NO c1034a34”. This error is somewhat misleading because we’re not REALLY trying to find a responsive Exchange 5.5 server, but rather, an SRS that acts just like an Exchange 5.5 directory.

If you run into this error, it means that the Exchange System Manager UI was unable to connect to the particular SRS that owns the naming context of the target site. You will probably need to confirm that this SRS is running, that there’s no firewalls blocking RPC communication between the ESM GUI and the SRS, etc. If you cannot connect to this SRS directory with 5.5 Admin from the machine running the ESM GUI, it is unlikely that the ESM GUI will be able to connect to do the prerequisite check!

But let’s go back even one step more. What if you are trying to run through the prereq check yourself, manually, and you can’t find the proper Config_CA listed in Step #2? If you’re using the ADC management UI to go through each listed Config_CA, it’s possible the Config_CA you’re looking for is not listed!

Each connection agreement is associated with one-and-only-one ADC server. There’s an attribute called msExchHomeSyncService stored on each connection agreement to link that connection agreement back to its owning ADC. It’s this association that causes each connection agreement to be listed subordinate to a particular ADC server in the ADC management GUI.

Here’s where it gets a little dicey. If you remove the ADC service from a server while it still has connection agreements associated with it, these connection agreements will become unassociated with *ANY* ADC server… they become “orphaned”, essentially. If you look at the msExchHomeSyncService attribute on such a CA, it will be blank. This means when you look through the list of all CAs in the ADC management GUI, you will not even see this CA listed. There’s a bit more info on this whole situation and how to recover from it in KB.319486.

Note that if you end up with the owning SRS for the target site as an orphaned SRS, you have bigger problems than the failure of the DS/IS prereq check! You will definitely want to clean up the ADC to make sure all connection agreements are properly in place, and that all SRS replication functionality is restored before attempting to move mailboxes cross site! Be sure to carefully follow the steps in KB.319486 and maybe rerun the ADC Tools (in the ADC management GUI) for good measure before attempting such moves.

Cross site moves in Exchange Native Mode

evand — Mon, 24 Jan 2005 22:32:00 GMT

Since I’ve posted a bunch of information on cross-site moves in mixed mode (using the Exchange 2003 SP1 feature), I’ve also had a number of folks ping me with questions about cross-site moves in Exchange native mode. There are a couple of points about these sorts of moves that are useful to understand:

Cross AG moves in native mode (intentionally) don’t change the LegacyExchangeDN

When you move mailboxes cross-site in mixed mode with the SP1 feature, part of the process (described in a bit more detail here) is to actually create a new mailbox in the target site. This new mailbox in the target site will have a new Object Distinguished Name in 5.5, which means it will have a different LegacyExchangeDN in the AD. Because it has a different LegacyExchangeDN in the AD, Outlook will no longer 100% associate any existing profile with the correct user — thus, why you have to run ExProfRe after doing cross-site moves in mixed-mode.

When you move a mailbox cross-site in native mode (cross-AG, we should actually say to be technically correct), the process is much more simple. We don’t change the LegacyExchangeDN on a mailbox/user during a native mode move. Since there’s no 5.5 in a native mode organization, there’s a line of thinking that LegacyExchangeDN is not important to the server anymore. This is sort of true — the Exchange server won’t care that you’re now in an AG that doesn’t match your LegacyExchangeDN, for instance — but it isn’t 100% true.

For instance, Outlook still uses LegacyExchangeDN. It uses it for things like Free/Busy updates.

Here’s an example scenario:
Two sites: Site1 and Site2. In Exchange native mode. User1 starts out in Site1 (Legdn: /o=Org/ou=Site1/cn=Recipients/cn=User1). User1 is moved cross-AG into Site2. Although all of the other Site2 mailboxes have LegacyExchangeDN of /o=Org/ou=Site2/cn=Recipients/cn=<useralias>, User1 retains the same LegDN after the move.

This can cause problems with Free/Busy updates

If you’ve moved mailboxes cross-AG as in the above Exchange native mode example, Outlook will still try to publish its Free/Busy updates to the system folder for the old site (“Site1”, in our example). This isn’t a problem — all free/busy updates can continue to work just fine in this case, as all references to the moved mailbox will still point to the old LegacyExchangeDN. Where it can be a problem is if you don’t realize this and decommission the source site (Site1).

If you finish migrating all of the mailboxes and get rid of Site1, you will need to make sure you have pushed a replica of the Site1 “SCHEDULE+ FREE BUSY” system folder to a site that your moved mailboxes have access to (probably Site2). It may look a little strange, putting a replica of the Site1 F/B information onto a server in Site2, but if you don’t do this your mailboxes will not be able to update Free/Busy information and when Site1 disappears you’ll have hash-marks as Free/Busy information for all of the users moved from Site1!

It can also cause problems anywhere else you’re using LegacyExchangeDN

A good example of this is Recipient Policies. If you’ve defined recipient policies based on LegacyExchangeDN (and in Exchange mixed mode, that’s exactly how recipient policies are defined!), these will not work exactly as you might like when you start to comingle non-matched LegacyExchangeDNs within the same site. If you’ve moved mailboxes from Site1 to Site2, you’ll find that your LegacyExchangeDN-based recipient policies that used to map one-to-one with your 5.5 sites will not consider the mailbox to belong to the Site2 recipient policy. Be sure you consider if there is anywhere else you’ve explicitly tied things to LegacyExchangeDN…

You don’t need to run ExProfRe

Another thing to consider is that if you don’t change the LegacyExchangeDN on these moved users, there’s nothing to update in the Outlook profile. There’s no need in that case to run ExProfRe, and if you do it will just tell you there’s nothing to update.

KB.841765 Ex55 Post-Sp4 hotfix re-released

evand — Fri, 17 Dec 2004 05:03:00 GMT

The "Exchange 5.5 Post-SP4 Rollup" (KB.841765) containing the DS/IS Hotfix required for Exchange 2003 SP1 Site Consolidation work (see my earlier posting) has been updated as of December 16, 2004. If you've been looking for it, you can now download the updated version here.

Cross-AG moves and ADC version upgrades

evand — Mon, 22 Nov 2004 20:31:00 GMT

If you've tried to do Exchange 2003 SP1 cross-AG mixed-mode moves, you've (hopefully!) realized that you need to meet a bunch of prereqs before you can proceed. The Site Consolidation Overview KB (KB.843104) lists these various prereqs, but here is the short version again for reference:

Upgrade all ADCs to Exchange 2003 SP1 (earlier blog posting)
Apply DS/IS hotfix to all 5.5 (KB.841765 and earlier blog posting)
Ensure your "target" server is running Exchange 2003 SP1

The second one has been covered pretty extensively in my earlier blog posting, and the third one is sort of a no-brainer. That leaves the first one: Upgrade your ADCs to SP1.

In the earlier blog posting, I said you had to do this ADC upgrade. But what if you are fairly certain you've upgraded them all and the cross-AG moves are still blocking you with an error message like: "Your organization has at least one Active Directory Connector that is not Exchange 2003 SP1 or later. Cross administrative groups moves will be blocked until all Active Directory Connectors have been upgraded to Exchange 2003 SP1 or later."

Let's talk about what is really being checked during this prerequisite process. First of all, the info we check regarding the ADCs installed and their version may have very little to do with what's really up and running in your environment. This info is all stored in the AD. For example, say you install an ADC on a test server in your production environment and then fdisk this server when you're done. Poof, lingering ADC object in the AD that most likely does not identify itself as an E2k3 SP1 version.

Each ADC defined in the AD is stored at a location like this:

CN=Active Directory Connector (SERVER),CN=Exchange Settings,CN=SERVER,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=DOMAIN

You can find your way to the ADCs defined in your organization by using AD Sites and Services snap-in, or perhaps (and carefully!) with ADSIedit snap-in. But the easiest way may be to use the LDIFDE tool to simply scan the whole configuration naming context for these objects.

Use a syntax like this (you'll have to change the domain info in red for your environment, of course):
LDIFDE -f out.txt -d "cn=configuration,dc=domain,dc=com" -r "(objectClass=msExchActiveDirectoryConnector)" -l versionNumber

This will find all instances of msExchActiveDirectoryConnector objects underneath the configuration container and will return their versionNumber.

In many cases, at this point it'll be very easy to see that this out.txt file lists ADCs that you weren't expecting or that no longer exist. If this is the case, either upgrade them to E2k3 SP1 version or remove them from the AD.

But suppose there are only ADCs at this point that you think have been upgraded. The prerequisite for cross-AG moves checks this versionNumber value to see if it indicates that it's an SP1 version, so here are the two most common versions you'll see listed:

16973842 = Exchange 2003 ADC RTM (initial release of 2003)
16973843 = Exchange 2003 ADC SP1 version

If it's the RTM version, clearly it wasn't upgraded successfully. Have another look at my earlier blog posting to make sure you followed the correct steps. If all of the ADCs listed show the SP1 versionNumber, then you've probably got an AD replication problem and the prereq is talking to a different DC than your LDIFDE process.

So what if the "stub" objects never go away?

evand — Mon, 06 Sep 2004 03:31:00 GMT

A while back, I wrote a blog post about verifying that you are ready to remove the source site after undergoing the process of SP1 mixed-mode site consolidation. See here for that intro. Since that post, I've run into a few questions about what to do if these validations never pass.

Suppose you keep checking for the stub objects to disappear in the source site by doing a CSV export and looking for the absence of "X500:ADCDeleteWhenUnlinked" proxy addresses. And suppose these addresses keep turning up each time you check. This means that the stub objects are not being cleaned up -- probably for some good reason.

Here's a deeper look at what has to happen to facilite the cleanup of the stub objects in the source site after the object has been moved cross-site. Suppose we've moved a mailbox:
1) After the mailbox data has been moved to the target site, attributes are changed on the mailbox directory object in the 5.5 source directory and in the AD to indicate the move.

5.5 Source object (soon to become the "stub" object) has:
- its proxy addresses removed
- its alias changed to a random GUID value
- HomeMdb and HomeMta changed to the new server location
- Adc-Global-Names flags updated to indicate cross-site move
- two X500 addresses added (X500:ADCDoNotReplicate and X500:ADCDeleteWhenUnlinked)

AD object is also updated (it will be the "authoritative object" during the cross-site move)
- legacyExchangeDN changed to point to the new site (this is what really defines the move to Exchange)
- msExchADCGlobalNames flags updated to indicate cross-site move
- new X500 address (for the old legacyExchangeDN) added to the existing proxy addresses
- HomeMdb, HomeMta, and msExchHomeServer attributes updated to the new server location
- Each DL this mailbox is a member-of is "touched" to ensure it will replicate the membership back to 5.5

2) The ADC will shortly notice that both the 5.5 source-site object and the AD object have changed

3) ADC will try to replicate the changed 5.5 mailbox object from the source site into the AD. This will be prevented by the "X500:ADCDoNotReplicate" proxy address. This is good. Otherwise the ADC would strip off all the proxy addresses, etc (the changes we made to the 5.5 source-site object during the cross-site move).

4) ADC will reverse direction and try to replicate the object from the AD over to 5.5. It will notice that the msExchADCGlobalNames flag indicates a cross-site move for this object and will reject any object match back to the source-site 5.5 object. ADCwill then need to find a new match, and because the legacyExchangeDN now points to the target site, ADC will create a new mailbox directory object in that site.

5) At some point (there are a number of complicated timing issues we'll avoid for this discussion), we'll also replicate the DLs from the AD back to their home site. Remember, we touched these DLs to ensure they would replicate. ADC will bring across the DLs and their respective memberships -- which now includes reference to the removal of the source-site object, and the addition of the target site mailbox object.

6) 5.5 Dirrep will run between the various sites in the 5.5 organization. The new mailbox object in the target site will eventually show up in the read-only naming context of the source site (All 5.5 directories with dirrep configured have full visibility of the whole 5.5 org). The DLs and their updated membership will also make it around the whole 5.5 organization after replication.

Status check: So, after all this ADC replication and Dirrep we should have a new mailbox directory object in the target site, an updated AD mailbox directory object, and a "stub" object in the source site. We only retain the "stub" object so that we don't mistakenly remove it from its DL memberships. We can't remove this object until we're SURE that all DLs have been updated to replace the "stub" object with the new mailbox object. Once we've updated all of the DL memberships, we're ready to go onto the next step. This is probably the part of the process where most people get stuck -- waiting on dirrep or ADC replication to finish.

7) The ADC runs a process every 12 hours to do miscellaneous cleanup. One of the new additions in the Exchange 2003 SP1 ADC is the addition of a "stub object cleanup". ADC checks every 12 hours (or you can force it to happen immediately with a "replicate now" action on the CA) for objects in the source site which are stamped with X500:ADCDeleteWhenUnlinked proxy address. If any are found, they are inspected to see if they belong to any DLs. If they do not (meaning DLs have all been transitioned off successfully), ADC also checks to see if there is an object in the 5.5 directory that has the X500 address of the stub mailbox object (indicating that the new mailbox object has replicated in over 5.5 dirrep). If the ADC finds another object with the X500 address, deletes the "stub" object since the process has been completed.

That's a bunch of steps, and it really only covers the most simple ordering. All of these steps are asynchronous and loosely coupled, so it's quite possible it might take several cycles to get all of the steps completed in the right order.

Hopefully that all makes sense. Perhaps I'll drill back into some of it in more detail for a future post if there's any interest.

Bonus tip: If you turn off ADC deletions (and you surely haven't done this now, have you? It's generally a bad idea, in my opinion), the "stub" object won't be deleted in step #7. Rather, the X500:ADCDeleteWhenUnlinked proxy address will be removed. This means that you will find lots of hidden "stub" objects in the 5.5 source directory, but they will not show up in the CSV export looking for this proxy address. You will see only the X500:ADCDoNotReplicate proxy remains on these objects. These are safe to delete, as they're only still retained because Deletions are turned off on the CA.

Update May24,2005: An important point I did not emphasize enough in this original posting is the importance of proper ADC connection agreements. Have a look at this later post for a bit more detail on that topic.

Do I really need to run the Exchange/Outlook Profile Redirector (ExProfRe)?

evand — Tue, 24 Aug 2004 19:40:00 GMT

Recent newsgroup posting caught my eye:

I've done cross sites moves today from 5.5 servers to 2003SP1 servers. The documentation says I need the exprofre.exe tool to update the Outlook profiles. But I don't need it, the profile connect automatically to the new server in the other administrative group/site. Why should I use it?

This is a valid question, and one I've been asked a bunch of times while teaching the SP1 Site Consolidation training internally at MS.

The unfortunate truth is that as part of the cross-site mailbox move process, the HomeMDB and HomeMTA attributes are updated on the stub user (see this posting for a mid-level view of what goes on during the move mailbox process). When these attributes are updated, this means that the next time your Outlook client tries to connect to your original 5.5 mailbox server, you will be redirected to your new mailbox server just as though the mailbox was moved within the site. This is totally expected behavior.

So why isn't this ok? Why do you still need to run the Exprofre.exe tool?

Well, remember -- you HAVEN'T just moved the mailbox to another server within the site. In fact, you've move the mailbox data to a completely different 5.5 site, and in doing so you have marked the original site's mailbox directory object as a "stub" object. Further, you have created a totally new mailbox directory object in the target site; a directory object that has a new and distinct "object distinguished name" (legacyExchangeDN in Exchange 200x terms). This is a totally different user as far as Exchange 5.5 and Outlook are concerned.

Outlook is perfectly capable of sorting out an intra-site mailbox move, changing the Outlook profile properties that related to the mailbox server location. It is not, however, capable at present of figuring out that not only has the mailbox data moved, but that the mailbox is no longer associated with the same directory object! After a cross-site mixed-mode mailbox move, the Outlook profile will be updated to point to the new mailbox server location, but the Outlook profile will *NOT* be updated to reflect the new mailbox directory object. Throughout the Outlook profile will be references to the old (now "stub") mailbox directory object in the original 5.5 site. Any time Outlook tries to reference these properties in the Outlook profile, the wrong DN will be returned.

That's the primary thing Exprofre fixes -- the update of the old mailbox directory name to the new mailbox directory name. But it also cleans up a handful of other stuff as well. For more information on the Exprofre utility, what will be broken if it's not run, and what it changes in the Outlook profile please see KB.873214.

Note: There's an even shorter answer to this question if you don't care so much about the technical aspects. Yes, you should absolutely run Exprofre on affected Outlook profiles after cross-site mixed-mode mailbox moves. If you do not either run this utility or recreate the profile as new, the profile will be in a state that has not been tested by Microsoft and results may be unpredictable!

Should I use the SP1 cross-site moves?

evand — Tue, 24 Aug 2004 19:03:00 GMT

First, my apologies to the folks I'd planned to grab lunch with while I've been out in Redmond these past few weeks. The project I'm working on has been keeping me much busier than I had expected. Maybe next time!

Onward -- Someone commented a few weeks ago on an earlier post:

a question about site consolidation: I have a couple 5.5 single-server sites in my org. Is it best to use the cross-site move method described in your webcast, or can I simply install the 2003 server in the single-server site, move all objects, then once in native mode move the server to the appropriate Admin Group?

Well, you can't move the servers between Admin Groups, even if you're in Exchange Native Mode. But this is a pretty good question, really, if you look at it from the "moving objects" perspective. The new SP1 Site Consolidation feature is great, but it can also be pretty complicated to execute, even with the appropriate planning. Maybe the old way of doing is still suitable in some cases?

The two generally available consolidation methods are:
1) SP1 Site Consolidation - Do planning, meet prereqs, use the new tools to migrate while still in mixed mode.
2) Native mode Site Consolidation - Do planning, meet prereqs, switch to native mode and then move stuff.

Ok, so that's a bit of an oversimplication. But at the core, the decision is one each customer must make. Prior to SP1, there was only the 2nd option.

The most obvious characteristics of this second option are:
1) Have to install additional servers (need an E2k/E2k3 server in each 5.5 site). This is possibly not a small problem -- if you have hundreds of 5.5-only sites, adding an E2k/E2k3 server into each is HUGE!
2) Your mixed-mode infrastructure may become VERY complicated. Remember, you'll be administering a parallel Exchange 5.5 and Exchange200x/AD environment right through until you go native mode. This may have large admistrative or bandwidth implications.
3) Once you've removed all 5.5, followed by removing all SRS, etc and switched to native mode it's actually fairly easy to do your consolidations. You can use the move-mailbox GUI in native mode to move mailboxes between administrative groups. Public Folders and DGs/Contacts are likewise no longer bound by the AG once you're in native mode.

For smaller installations of Exchange 5.5 (say, a handful of sites), this might make the most sense. The planning is a bit less, and the prereqs a bit easier to meet.

But for larger Exchange installations, it becomes more difficult to justify implementing additional servers in remote locations right as you're being asked to consolidate existing servers. If this is the case in your environment, the SP1 Site Consolidation feature may be for you.

How can I upgrade the ADC to SP1?

evand — Thu, 22 Jul 2004 18:10:00 GMT

In order to to Exchange 2003 SP1 Site Consolidation (ie - cross-site/cross-administrative group moves), we require that the Active Directory Connector (ADC) be upgraded to Exchange 2003 SP1 level. In the webcast I did on Tuesday, I talked a bit about why we have this requirement, but the short version is: if your ADC was not SP1 or better, it would munge all of the great “special-case-for-site-consolidation” work that was included in the SP1 ADC.

The question occasionally arises then: how do I get the ADC to SP1 so I can get past this prereq? Here are a couple of facts you may want to be aware of:

The ADC does not automatically get upgraded to SP1 when you run the Exchange 2003 SP1 “Update.exe“
There is no “Update.exe“ for ADC
ADC is actually slipstreamed in each service pack (meaning you don't have to install the Exchange 2003 release version and then upgrade it if you're starting fresh with Exchange 2003 SP1 ADC)

The steps to get the ADC upgraded are covered as “Method #2” in KB.841667, but here they are modified for SP1:

Insert the Exchange 2003 SP1 CD-ROM into the CD-ROM drive (or navigate to the downloaded and extracted files) on all domain computers where any version of the ADC program that is earlier than the Exchange 2003 SP1 version is installed.
Open the ADC\i386 folder, and then double-click Setup.exe.
When the Active Directory Connector (ADC) Installation Wizard starts, click Next.
Click Reinstall to upgrade to the Exchange 2003 SP1 version of the ADC program.

The important consideration here is the option to “Reinstall“ within the ADC setup. Don't choose Remove All or the ADC will be uninstalled (and your ADC Connection Agreements will be disconnected as described in KB.319486).

Edited (Nov29,2004): I notice in reviewing the referrer logs that a lot of folks are hitting this page looking for information on Exchange 2003 slipstreaming. Sadly, I have to report that while SOME of the components (like the ADC) are slipstream - meaning that you can just install it freshly with the SP1 version and be running SP1 ADC -- the core Exchange 2003 product cannot be slipstreamed. In order to get to Exchange 2003 SP1, you have to install Exchange 2003 and then upgrade it to SP1 version. Sorry!

I also talked about KB.841659 in my webcast on Tuesday. It's now been made available:

KB.841659 - Overview: Issues to know about during Site Consolidation to an Exchange Server 2003 Service Pack 1-based site - This KB is a monster (probably 30 pages printed) article that covers the known issues for each of the object move types (public folder, mailboxes, distribution lists, custom recipients). Very comprehensive and useful in preparing both yourself and your users for cross-site moves.