Petergal's SBS Blog

Providing an insight to the day to day technical issues a Microsoft SBS Support Engineer faces.

Blogs

The rest of the story...the migration completed...(long post)

  • Comments 3
  • Likes

Well, I have procrastinated long enough on wrapping up the migration story.  Here it is...

----------------------------------------

I planned this process as much as I could, most went as planned, some oversights.  Note that prior to my steps below, These steps were performed about a week prior (transferred the FSMO roles, exported the public certificate).  

 

Here is what I did:

 

  1. I copied my most recent SBS Backup file from a Windows XP machine back to the local SBS box on a separate partition.  I did this in case I ran into a driver issue (the network cards).  I could always restore the Windows directory to an alternate location, then point the hardware wizard to c:\restored\winnt\inf and it should pick up the drivers
  2. Made sure my backup of the replica domain controller was successful
  3. Uninstalled WSUS and deleted my WSUS policy.  I did this because I just got a copy of SBS R2 which includes WSUS.  I also needed the space on the server to copy the backup file local to the sbs box (step 2 above).
  4. Opened Active Directory Users and Computers on the SBS Box and did a search for users with Exchange mailboxes, highlighted them all, right click, Exchange tasks, delete Exchange attributes.  You can't uninstall Exchange if there are users who have mailboxes.
  5. Went into Active Directory Sites and Services and did a Replicate Now for both connections
  6. Ran the SBS integrated setup, chose to REMOVE Exchange
  7. Rebooted the SBS box
  8. Ran DCPromo on the SBS box and chose "remove" Active Directory
  9. Got an error regarding "timeout" for the Netlogon service, continued...
    Got a message stating that Active Directory was successfully removed and prompted for a reboot, rebooted Server.
  10. While Server was rebooted, hopped over to Win2kdc and performed the following:
    • Ran NTDSUTIL to see if the server was gone, it was.  Just looking around…
    • Went into Active Directory Sites and Services -> Sites -> Default-First-Site-Name -> Servers and right-clicked on Server and chose DELETE.
    • Deleted all the Exchange attributes in Active Directory Users and Computers (click on View -> "Advanced Features".  Under Company.Local, located and deleted Microsoft Exchange System Objects.
    • Deleted Exchange attributes in Active Directory Sites and Services.  Opened AD Sites and then clicked on Active Directory Sites and Services (win2kdc.company.local) and then clicked View -> "Show Services Node".  Under Services, located and deleted Microsoft Exchange.
    • Opened DNS on Win2kdc, removed server.company.local from the Nameservers tab on each of my DNS zones (both forward and reverse zones).
    • Stepped though each zone and deleted any record for server.company.local and/or192.168.50.2.
    • Rebooted Win2kdc for good measure.

 

At this point, I simply have a single domain controller (remember this domain controller is in Virtual Server on a WinXP Pro box) with no trace of Exchange installed.

 

Now off to install my clean install of SBS.

 

  1. Booted the old box from the SBS media and kicked off the setup (the Windows portion)
  2. Unplugged the network cable from the cable modem 
  3. Deleted the C partition, created new, format NTFS, off we go...
  4. At this point, I started copying the other SBS CD's to my Windows XP workstation so I could complete the rest of the SBS setup across the network, not having to flip CD's.
  5. Typed in the product key and then a servername and Administrator password.  The servername I chose was "Apollo".  Continued setup...
  6. While SBS was installing, took a quick look around on Win2kdc.  Checked Event Viewer and was presented with this error:

Event Type:  Warning

Event Source:  NETLOGON

Event Category:  None

Event ID:  5781

User:  N/A

Computer:  WIN2KDC

Description:  Dynamic registration or deletion of one or more DNS records associated with DNS domain 'DomainDnsZones.company.local.' failed.  These records are used by other computers to locate this server as a domain controller (if the specified domain is an Active Directory domain) or as an LDAP server (if the specified domain is an application partition). 

 

Possible causes of failure include: 

TCP/IP properties of the network connections of this computer contain wrong IP address(es) of the preferred and alternate DNS servers
Specified preferred and alternate DNS servers are not running
DNS server(s) primary for the records to be registered is not running
Preferred or alternate DNS servers are configured with wrong root hints
Parent DNS zone contains incorrect delegation to the child zone authoritative for the DNS records that failed registration  
 

USER ACTION 

Fix possible misconfiguration(s) specified above and initiate registration or deletion of the DNS records by running 'nltest.exe /dsregdns' from the command prompt or by restarting Net Logon service. Nltest.exe is available in the Microsoft Windows Server Resource Kit CD.

 

Don't really care about that one! 

 

SBS setup is continuing on fine...

 

  1. The Windows 2003 portion of setup completed, statically assigned an IP address to the nic, pointed him to Win2kdc for DNS, enabled Remote Desktop, disabled the Windows Firewall on the nic and then did an RDP from the Windows XP workstation to Apollo. 
  2. I am now "joining SBS to an existing domain" per this article. Kicked off DCPromo.  DCPromo completed and required a reboot.  I rebooted and walked away...came back to a logon prompt and logged on.  Walked away again and came back to the SBS desktop...the actual fully installed SBS desktop from earlier in this process!  $(#*$*(#???  I have a hardware mirror...did the mirror break?  So I started poking around on the "SBS" box.  A quick inventory of the contents of the root of all the drives showed me that my Apollo install is actually on the E drive and the SBS install (Server) shows as on the C drive.  Nice!  What happened was that I accidentally installed to an IDE drive.  All my partitions/drives are very close in size.  No worries, clean install means clean install.    Anyway, what to do next.  The phone just rang...my house is for sale and it is going to be shown from 12:30 to 1:30 today!  Now I have about 2 hours to get this thing back on track!  When it rains, it pours!  Oh, I wonder how Active Directory is responding to this Domain Controller being back on the network?  Ok, so I unplugged the IDE drive and kicked off another setup.  I deleted all partitions and created a single partition and setup continued.  While setup was running, I hopped over to Win2kdc and ran through the NTDSUTIL article again and removed APOLLO from Active Directory Users and Computers and Active Directory Sites and Services (basically the same thing I did above for “Server” but this time for “Apollo”).  I then stepped through DNS again on Win2kdc and deleted all references of APOLLO. 
  3. Apollo setup completed, rebooted and it is now just running Windows 2003 Server as a stand-alone server.  I set the IP address to static, pointed him to Win2kdc for DNS and ran DCPromo (following this article again).  It prompted for a reboot, which I did.  This time when I booted up, it was to the right place.  Now on with my plan. 
  4. I disabled the Windows Firewall on Apollo, turned on Remote Desktop and logged off.  I then logged on via Remote Desktop from the Windows XP machine (did I mention that my server Apollo does not have a mouse?). 
  5. Once logged on, I installed DNS on Apollo, set Apollo to point to himself for DNS, waited for all the Active Directory Integrated DNS zones to show up on Apollo (about 30 seconds)
  6. Made Apollo a Global Catalog server and rebooted Apollo.  Apollo is now a "replica" DC in an existing domain.  I am now waiting for the events to show that Apollo is successfully a Global Catalog server.  Waiting...nothing.  I go check to see if Apollo is set to be a Global Catalog server, he is not, checked the box again and rebooted Apollo again...now I have the event I am looking for:

 

Event Type:  Information

Event Source:  NTDS General

Event Category:  Global Catalog

Event ID:  1869

Computer:  APOLLO

Description:  Active Directory has located a global catalog in the following site.

 

Global catalog:

\\apollo.company.local

Site:

Default-First-Site-Name

 

  1. Even after seeing the event, I check replication manually.  I create a user in Active Directory on Win2kdc and then check Active Directory on Apollo, there he is!  I then delete the user in Active Directory on Apollo and check Active Directory on Win2kdc, he's gone.  I then open up Group Policy Management on Apollo and change a simple policy (the Windows Firewall Policy, I set it to "disabled") and then opened up Group Policy Management on Win2kdc and the change was replicated.  I then did a GPUPDATE /FORCE on one of my clients and waited for the SCECLI event 1704 that says "policies were applied successfully", no go.  What gives?  Well, back to my original configuration the SBS box was the DHCP server handing out addresses pointing to himself for DNS...that is now a problem because my clients are all DHCP clients.  I set my box up to have a static DNS server pointing to Apollo and restarted NETLOGON on the client and did an ipconfig /flushdns on the client and then repeated the GPUPDATE /FORCE and the Windows Firewall Policy was gone (meaning I can turn off/on the Windows Firewall on the client)!  This is good!
  2. I then removed the "Global Catalog" setting on Win2kdc and then rebooted Win2kdc.   The it's now 11:00 AM, about 30 minutes until I have to step away to get the house cleaned up for the 12:30 showing!!  The phone rings, one of my "users" wants to know when email will be back up...  <sigh>. 
  3. Now I transferred the FSMO roles to Apollo, forced replication and then kicked off the SBS setup on Apollo, acknowledged the warning about the DS Restore Mode password being synced with the Domain Administrator account, hit next, prompted for a CD, mapped a drive to the Windows XP box that has all the install files for SBS, pointed SBS setup across the network and setup continued...got to the component selection screen, accepted the defaults for what to install and where to install it and continued...left the house so it could be "shown".  Back an hour later...  The only hiccup was that the SBS setup barked for the Outlook CD, which I manually browsed to.  Setup completed.  Active Directory is intact, life is good.  Now on with the the rest of the setup. 
  4. Installed Exchange 2003 SP2
  5. Visited Windows Update, installed all critical updates...
  6. Time to restore Exchange.  I dismounted both the Public and Private stores, marked each to be "overwritten by a restore" and deleted the contents of c:\program files\exchsrvr\mdbdata and kicked off NTBackup.  Chose the Information Store and hit go.  It failed immediately and generated the following error in the Application Log:

Event Type:  Error

Event Source:  MSExchangeIS

Event Category:  Exchange Backup Restore

Event ID:  9635

User:  N/A

Computer:  APOLLO

Description:  Failed to find a database to restore to from the Microsoft Active Directory.

 Storage Group specified on the backup media is 581cb7ee-5fec-4d93-8f71-dcfe55a73319.

 Database specified on backup media is Mailbox Store (SERVER), error is 0xc7fe1f42.

 

Hmm…the storage group names do not match.  New = Mailbox Store (Apollo), Old = Mailbox Store (Server).  I renamed Mailbox Store (Apollo) to Mailbox Store (Server) in Exchange System Manager -> Servers -> First Storage Group -> Mailbox Store (Servername) and kicked off the restore again.  This time it proceeded and completed successfully.  This was an oversight regarding the default behavior of an Exchange install using the SBS integrated setup.  I *could* have ran the Exchange setup manually from the SBS media and it would have prompted me for the site/org name.  Exchange data has been restored (minus Public Folders, don’t have any data).

 

  1. Now time for ISA2004.  I browsed across the network and kicked off the ISA2004 install from the Premium Technologies CD, completed, ran CEICW, chose mail.company.com, completed. 
  2. Created a rule in ISA to accommodate DNS hosting <don’t host your own DNS>
  3. Imported the original certificate into the local certificate store.  Start -> Run -> MMC -> Personal -> Action -> All Tasks -> Import.  Remember I exported the original certificate prior to the migration.
  4. In ISA, changed the certificate from the new mail.company.com to the original certificate I just imported.  This is done in ISA -> “Any SBS Web Publishing Rule” -> Properties -> Listener -> SBS Web Listener -> Properties -> Preferences -> Select.
  5. What's left...tested mailflow from Hotmail and Microsoft to my domain, success.  Ok, we look golden. 
  6. I checked my user that uses a SmartPhone.  The SmartPhone freaked a couple of times, I deleted the settings and typed them back in on the device, works. 
  7. Now on to the RPC over HTTP computer.  Here is where I discovered another error in my ways.  I went to great lengths to ensure the certificate was the same so I would not have to go re-add the certificate to the RPC over HTP clients, what I forgot is that since I changed the servername from SERVER to APOLLO, they would all have to be touched anyway.  My worst case I thought would be that I would delete and recreate the profile in Outlook.  Oh well, live and learn.  Deleted and recreated a new Outlook profile, works.  I did this on my 2 RPC over HTTP clients.
  8. For grins, I sent another email just to test and got a lovely NDR stating that the recipient’s mailbox was full!  The default behavior of an SBS install is to limit the mailbox size of all the users to 200 MB.  My mailbox is a GIG and another one is 700 MB.  It was interesting that my original tests from Hotmail and Microsoft came through.  I went to my account and the other account in Active Directory (properties of the user -> Exchange General -> Storage Limits and unchecked “Use mailbox store defaults”) and overrode the default limits.  This could have been a MAJOR issue and COULD have resulted in data loss.  Hey, did you get that email I sent???  Nope!
  9. I then checked my daughter’s mailbox on her workstation.  She was hanging out and looking over my shoulder while I was working through this process.  She is getting mail about the Nigerian prince and his fortune as well as some choice emails about Viagra.  This really pissed me off!  I honestly had to explain the Nigerian thing to a 9 year old.  Thank goodness she did not see the Viagra spam.  I had not implemented any of the anti-spam features of Exchange.  I implemented my settings again (IMF/Connection Filtering, etc) and then deleted the email out of her mailbox.
  10. I then implemented the steps in this article to allow Microsoft Update to automatically update the signatures for the Intelligent Message Filter.  A quick trip to Microsoft Update and “custom” presented me with the IMF updates which I installed. 
  11. Tried to browse the web from her Windows XP pro client, failed!  “Cannot find server or DNS error”.  On her computer, Internet Explorer is NOT set to use a proxy server and the ISA Firewall Client is installed.  I noticed the Firewall Client was “banged” in her System Tray.  She is running the ISA2000 Firewall Client.  Doing an “update now” was successful and removed the “bang”.  Trying to browse resulted in the same error and the Firewall Client “banged” out again.  Remembering the default behavior of an ISA2004 “clean install” as opposed to an upgrade from ISA2000, “Allow non-encrypted Firewall client connections” is UNCHECKED on a clean install of ISA2004 and it is CHECKED on ISA2000 -> ISA2004 upgrades (on SBS anyway).  In ISA -> Servername -> Configuration -> General -> Define Firewall Client Settings I CHECKED “Allow non-encrypted Firewall client connections” (my previous install was an SBS2000 -> SBS2003 upgrade thus the box was checked prior to the migration) and hit OK and then Apply.  Now she can browse!

 

At this point, I think I am back to a good working server/network.  It's about 5 in the afternoon.  Not too proud of the way things went for this migration. 

 

And now…the rest of the story…

 

Woke up on Sunday morning, had my cup of coffee and started the process over again!  I basically did the same steps as before but chose Server as the SBS servername.  No DCPromo down, no uninstall of Exchange.  I transferred the FSMO roles, booted Apollo off the SBS media and started a fresh install.  While this was running, I cleaned out Apollo from Active Directory using NTDSUTIL and manually deleting the Exchange objects from Active Directory Users and Computer as well as Active Directory Sites and Services.  I started at 7:00 AM and was complete at around Noon.  Much smoother experience.  The majority of the time was the install itself.  The only “unexpected” issue I ran into was the Anonymous Account’s password for IIS was out of sync with the password in Active Directory.  By default, the anonymous account is called IUSR_Servername.  In my case, IUSR_SERVER.  The symptoms were:  trying to browse Outlook Web Access from the server itself gave the NTLM popup logon – not the pretty OWA login page; providing the username/password resulted in a half-drawn OWA page (very similar to the issues resulting from Q831464 but I have SP2 for Exchange, so that’s not it).  As a quick test, I tried to browse http://server/tsweb and got a 440 Login Timeout error.  I use TSWeb all the time to test basic IIS connectivity.  It is a website installed by default on SBS with Anonymous Access turned on.  The 440 Login Timeout is usually the anonymous account password in IIS is out of sync with Active Directory. 

Here is how I fixed that:

  • Open Active Directory -> Users -> IUSR_SERVER -> Right Click -> Reset Password
  • Open Internet Information Services Manager -> Servername -> Web Sites -> Right click Web Sites -> Properties -> Directory Security -> Edit for Authentication and Access Control -> typed in the password that I set in Active Directory in the Password field.
  •  Click OK and then “select all” for Child Nodes.
  • Select All again for Child Nodes
  • Restarted the IISAdmin service

Wrapping up the migration, I made sure all the clients were pointing to Server for DNS.  Login scripts are running and policies are being applied.  Life is good again!

Here is a list of quick hits regarding this migration:

  • This may have seemed hard but it was really pretty easy.
  • Always test your solutions!  Know your processes!  I have walked many customers through each of the different aspects of a migration.  Putting it all together can be unique for each scenario.
  • When migrating with this type of method, always test replication throughout the process.  Replication should always be working. 
  • I messed with the clients “during” the migration, which was not necessary.  Replication between the servers is the key.  Clients may freak during the migration.  It is a waste to troubleshoot clients in the middle of the migration.  Get the AD stuff migrated between the servers, get DNS right on the server(s)...then start messing with the clients.
  • Did I mention DNS?  If DNS is wrong, Active Directory replication will be broken which will make the migration much more difficult
  • Have verified backups prior to starting any migration
  • Know the Exchange restore method that is specific to the migration scenario.  PST files, Offline Backups, Online Backups…all of which present different challenges/requirements when restoring.
  • Did I mention DNS?
  • Did I mention testing replication?
  • It was painful for my RPC over HTTP user that had a 700 MB mailbox.  She is accessing Exchange from 2 different computers with RPC over HTTP.  It took a full 2 days to sync both Outlook clients!  The user’s complaint was that “sent items” was not updated on each client (“Hey, I sent a mail from the office but when I go check “sent items” on my home computer, it does not show up…did it go out??”).  I should have advised her to leave Outlook open when she left the office (or wherever) to allow it to do a full sync.
  • Get your antivirus/anti-spam solution installed and up-to-date prior to putting the server (or the server’s services, like SMTP) live on the internet.
  • Remember the default configuration settings of an SBS install and what you have “tweaked”.  Mailbox limits in Exchange, IMF settings, IMF updates via Microsoft Update, Disk Quotas (defaults to 1 gb), ISA2004 setting changes, etc.

 

Have a good weekend!

 

Petergal

Comments
  • Very detailed description... I have accomplished approx 7 swing migrations since I hooked up with Jeff Middleton. I have refined his technique which parallels that which you have laid out (well, it should, shouldn't it???)

    My most recent two migrations were accomplished with the first step (moving the AD and DNS settings to a migration server, and then disjoining and seizing FSMO roles) was done remotely via a PPTP VPN. I didn't think that it could be done, but my second one was easier than the first. As always, the thing that trips me up the most is making sure that all replication (especially SYSVOL share) is accomplished. Really, the only problem that I have had to fix (sometimes at great time expense) is replication.

    Question (unrelated to migration): If a user has a notebook and spends a lot of time out of the office can he set up RPC over HTTP for his Outlook and float between being on the SBS LAN and not being on the LAN? Will OL2K3 sense being local and connect straight thru and not access through RPC?

    Thanks for your blog and the podcasts!

    Randy

  • PingBack from http://apollo440.musicnewsandviews.com/2006/05/02/the-rest-of-the-storythe-migration-completedlong-post/

  • Thank you for valuable information.