Ask the Core Team

DPM - How to Enable End-User Recovery using Windows 2008 Server

To enable the end-user recovery in DPM 2007 on a Windows 2008 Server you should click on the recovery tab and then under actions (on the left hand side) click on “Configure end-user recovery…” link.  This will launch a wizard to extend the 2008 schema so that end users can recover files.  However due to security changes in Windows 2008 Server you will get an error - "Active Directory Could Not be Found" from running this on the DPM server.

Ordinarily you would configure End-User Recovery beginning in the DPM UI, under the "Recovery" tab.

Click “Configure end-user recovery…”

Click on the configure active directly button

Enter the user name, password, and domain.

Click yes

Click OK - the process will error...

To install the necessary Schema extension log onto the Domain Controller and then map to the DPM installation directory on the DPM server, by default this will be C:\program files\Microsoft DPM\DPM\End User Recovery, and run the DPMADSchemaExtension.exe.

Click on run.

Click on yes

Enter the DPM server name

            Note: this is not the FQDN name of the server, but just the server name.

Enter your domain name.

            Note:  This will be the FQDN domain name so if your domain is yourdomain.local enter yourdomain.local.

This field can be left blank if the DPM server is in the same domain as the Domain Controller that owns the Schema master role.

Click OK.

Click OK.

Back on the DPM server, click on the recovery tab again, and then click on the “Configure end-user recovery…” link.

Notice that the configure active directory button is grayed out and that you can place a check mark in the “Enable end-user recovery” check mark button.

You will get a warning telling you that you must wait for a synchronization to take place before the setting change takes effect.  Click OK.

Author:  Keith Hill
Microsoft Enterprise Services

Installing SCVMM Portal to x64 Windows - Fatal Error during installation ID: 205

When installing the System Center Virtual Machine Manager 2007 Self Service Portal on a 64-bit version of Microsoft Windows, you may encounter a Fatal Error if IIS is configured to run in 32-bit mode.

This issue occurs when installing the SCVMM 2007 Self Service Portal on a 64-bit version of Microsoft Windows when IIS is configured to run in 32-bit mode. An example of the error message displayed by the SCVMM Self Service Portal Setup Wizard citing the “ID: 205, Fatal Error during installation” is shown below:

 



After the failed installation if you review the SelfServiceSetup.log which is located under
“C:\Documents and Settings\All Users\Application Data\VMMLogs” you would find line entries like the ones shown below:

MSI (s) (C8!44) [11:13:27:686]: Creating MSIHANDLE (169) of type 790531 for thread 4420
CAQuietExec: The error indicates that IIS is in 32 bit mode, while this
application is a 64 bit application and thus not compatible.

MSI (s) (C8:54) [11:13:27:748]: Product: Microsoft System Center Virtual Machine
Manager 2007 Self-Service Portal (x64) -- Installation failed.

MSI (s) (C8:54) [11:13:27:748]: Cleaning up uninstalled install packages, if any exist
MSI (s) (C8:54) [11:13:27:748]: MainEngineThread is returning 1603
MSI (s) (C8:08) [11:13:27:858]: Destroying RemoteAPI object.
MSI (s) (C8:40) [11:13:27:858]: Custom Action Manager thread ending.
=== Logging stopped: 4/23/2008 11:13:27 ===
MSI (c) (BC:28) [11:13:27:858]: Decrementing counter to disable shutdown. If
counter >= 0, shutdown will be denied. Counter after decrement: -1
MSI (c) (BC:28) [11:13:27:858]: MainEngineThread is returning 1603
=== Verbose logging stopped: 4/23/2008 11:13:27 ===

[4/23/2008 11:13:27 AM] Information : MsiInstallProduct returned 1603.
[4/23/2008 11:13:27 AM] Information : End install.
[4/23/2008 11:13:27 AM] Information :
**************************************************************************************
[4/23/2008 11:13:27 AM] * Exception : => Microsoft System Center Virtual Machine
Manager 2007 installation did not complete successfully.Review the error log for
information, and then try Setup
again.Microsoft.VirtualManager.Setup.Exceptions.BackEndErrorException: exception
---> Microsoft.VirtualManager.Setup.Exceptions.MsiInvokeException: Error in the application.

at Microsoft.VirtualManager.Setup.Wizard.Installer.Install()
at Microsoft.VirtualManager.Setup.Wizard.BackEnd.Install(Installer installer)
*** Carmine error was: InstallFailed (205); WindowsAPI
--- End of inner exception stack trace ---
at Microsoft.VirtualManager.Setup.Wizard.BackEnd.Install(Installer installer)
at Microsoft.VirtualManager.Setup.Wizard.VMInstaller.Install()
at Microsoft.VirtualManager.Setup.Wizard.ProgressPage.InstallVm()
at Microsoft.VirtualManager.Setup.Wizard.ProgressPage.InstallerThreadEntry()
*** Carmine error was: InstallFailed (205); WindowsAPI
[4/23/2008 11:13:27 AM] *** Error : Microsoft System Center Virtual Machine Manager
2007 installation did not complete successfully. Review the error log for
information, and then try Setup again.
ID: 205. Details: Fatal error during installation
[4/23/2008 11:13:27 AM] Information : Virtual Machine Manager Self-Service Portal
installation did not successfully install. All items that were copied during the
installation process have been removed, however some required prerequisite software
is still present on the machine. It is not necessary to remove the remaining
software before you run Setup again. But you can uninstall the prerequisite
software by going to Add or Remove Programs.

To resolve this and allow the Self Service Portal to install properly, we will need to issue a command to configure IIS to run in 64-bit mode. The reason for this is IIS 6.0 supports both 32-bit and 64-bit mode however it does not support running both modes at the same time on a 64-bit version of Windows. Before issuing this command however you must download and install both the .NET Framework 1.1 and the .NET Framework 2.0 on the server if they’ve not been already.

To resolve this issue the following command will need to be ran on the server. Please do take careful consideration in running this command as it could potentially break other websites/applications that are already configured and running under IIS on the server.
 

ASP.NET 2.0, 64-bit version

To run the 64-bit version of ASP.NET 2.0, follow these steps:

1. Click Start, click Run, type cmd , and then click OK.

2. Type the following command to disable the 32-bit mode: cscript
%SYSTEMDRIVE%\inetpub\adminscripts\adsutil.vbs SET
W3SVC/AppPools/Enable32bitAppOnWin64 0

3. Type the following command to install the version of ASP.NET 2.0 and to install
the script maps at the IIS root and under:
%SYSTEMROOT%\Microsoft.NET\Framework64\v2.0.50727\aspnet_regiis.exe -i

4. Make sure that the status of ASP.NET version 2.0.50727 is set to Allowed in the
Web service extension list in Internet Information Services Manager.

Note: The build version of ASP.NET 2.0 may differ depending on what the currently
released build version is. These steps are for build version 2.0.50727.

Additional information:

Download location for .NET Framework: http://msdn2.microsoft.com/en-us/netframework/aa731542.aspx

894435  How to switch between the 32-bit versions of ASP.NET 1.1 and the 64-bit version of ASP.NET 2.0 on a 64-bit version of Windows
http://support.microsoft.com/default.aspx?scid=kb;EN-US;894435

Author: Tyler Franke
Microsoft Enterprise Platforms Support

10 commonly asked questions or issues seen during Hyper-V beta

With the RTM release of Hyper-V just around the corner, I thought it would be a good idea to re-visit some of the top things we have seen customers encounter when installing and configuring Hyper-V for the first time and give pointers to resources we have available to eliminate those when you start rolling this out in production for the first time. I have taken the liberty of linking many of these questions/issues to blogs written by our program managers on the virtualization team. We did a previous post on their sites but I thought this would serve as a quick reference and a pointer to them at the same time.

1. You don't have mouse functionality in your VM. One of the most commonly asked questions during the beta, internally and externally had to do with lack of mouse support in a guest when its running. One of the main reasons we saw this happening was that people were remoted into the parent and then controlling the guest from there, that wont work in Hyper-V. More information about best practices can be found here:

http://blogs.technet.com/jhoward/archive/2008/03/23/controlling-vms-under-hyper-v-without-the-mouse.aspx

2. Hyper-V wont install. Another common question had to do with Hyper-V not installing at all. People would add the role and either would get errors post installation, prior to installation, etc. So, to revisit the core requirements for the Hyper-V role to be installed and functional on your machine, see the release notes once we RTM. Top issues here though were folks not enabling Virtualization and DEP in their system BIOS, not shutting down the parent completely at the end of installation but rebooting the parent instead, and not adding the proper entries to the BCD store on a Core installation (hint: rebooting twice takes care of this in Core).

3. Import isn't working within Hyper-V for your Virtual Server VMs. Another common question is what the Import Virtual Machine button does within the Hyper-V console. Many people, myself included early on, thought that it was there to import a .vhd from a Virtual Server installation. It's not. It's there to import an exported Hyper-V virtual machine. The proper way to import a previously used Virtual Server .vhd file is to create a new virtual machine and use the existing hard drive function during setup.

4. Hyper-V clustering: Clustering Hyper-V installations is new and different in Windows 2008. Rather than re-inventing the wheel here I strongly encourage you all to use the excellent whitepaper thats out on the external Microsoft site:

http://www.microsoft.com/downloads/details.aspx?FamilyId=CD828712-8D1E-45D1-A290-7EDADF1E4E9C&displaylang=en

5. Snapshotting in Hyper-V. Snapshotting confuses a lot of people because they liken them to un-do disks in Virtual Server. They aren't the same thing. Excellent references below to how snapshotting works in Hyper-V and how to get the undo functionality that you're used to are linked below.

http://blogs.technet.com/roblarson/archive/2008/05/15/getting-undo-functionality-with-hyper-v-snapshots.aspx

and

http://blogs.technet.com/roblarson/archive/2008/04/26/virtual-machine-snapshots-with-hyper-v.aspx

6. Virtual Networks. Additionally virtual networking has been totally revamped in Hyper-V and many dont understand the purpose of the individual networking options within the console. Linkage to a great article on them is below:

http://blogs.technet.com/jhoward/archive/2008/06/16/how-does-basic-networking-work-in-hyper-v.aspx

and

http://blogs.technet.com/jhoward/archive/2008/06/17/hyper-v-what-are-the-uses-for-different-types-of-virtual-networks.aspx

7. Using Hyper-V in Windows Server Core. Server Core is the new command line only installation option and its perfect for Hyper-V. For many people it can be hard to navigate and install new roles however. Below is a good link on installing Hyper-V in a Server Core environment.

http://blogs.technet.com/jhoward/archive/2008/03/30/how-to-add-the-hyper-v-role-to-a-windows-server-2008-server-core-machine.aspx

8. Remote Management. Managing Hyper-V installations, be they GUI or Server Core, is always a concern. An excellent two part article on this is located here:

http://blogs.technet.com/jhoward/archive/2008/03/28/part-1-hyper-v-remote-management-you-do-not-have-the-requested-permission-to-complete-this-task-contact-the-administrator-of-the-authorization-policy-for-the-computer-computername.aspx

and

http://blogs.technet.com/jhoward/archive/2008/03/28/part-2-hyper-v-remote-management-you-do-not-have-the-requested-permission-to-complete-this-task-contact-the-administrator-of-the-authorization-policy-for-the-computer-computername.aspx

9. Issues deploying virtual machines within a virtualized environment. I have seen this a lot internally, you have a RID/WDS server in a VM that you've used forever in Virtual Server. You decide to move that installation to Hyper-V and now you cant deploy new children VMs in Hyper-V. The problem here lies in the fact that you need to use the legacy network adapter rather than the synthetic device for the child VM you are trying to deploy to. We dont support PXE boot on the synthetic NIC in Hyper-V. You can add a Legacy NIC in the Add Hardware section of the Virtual Machine Settings.

10. One other virtualized hardware issue we have seen is SCSI boot. Or more specifically, the lack of it. We no longer support booting from the virtualized SCSI controller in Hyper-V. You can still use the SCSI controller for storage drives and you can utilize iSCSI or SAN storage as well, you just cant boot SCSI anymore.

Author: Joseph Conway
Support Escalation Engineer
Microsoft Enterprise Platforms Support

SharePoint Protection and Recovery using DPM 2007 - Part I

With SharePoint gaining popularity in the last year and the recent release of DPM 2007 which supports the protection of SharePoint this has become a hot topic in the last few months. The protection and recovery of SharePoint data is not small undertaking and there is quite a bit of flexibility with what you can recover. For this reason, let me caution you in that this can be a complex undertaking but once the methods are understood, its benefits far surpass the trouble of the initial learning curve.

So let’s start with the basics of what is required in order to protect SharePoint with DPM and we will expand from there. Over the course of this series, we will cover protection and recovery on several various levels, each of which will have it’s own nuances which dictate this breadth of coverage.

• Protection Requirements
• Recovery of a SharePoint Farm
• Recovery of a SharePoint DB
• Recovery of a SharePoint Object (Site, sub-Site, folder, file, etc.)

We begin with the Protection Requirements. Below is a list of what is needed to fully protect and recover SharePoint data.

• DPM 2007 only protects “Microsoft Windows SharePoint Services 3.0” and “Microsoft Office SharePoint Server (MOSS) 2007”. If this is not what is installed in your environment, now is a good time to look at upgrading.
• The SharePoint patch from 941422 must be installed on all servers. Install Knowledge Base article 941422, "Update for Windows SharePoint Services 3.0" (http://go.microsoft.com/fwlink/?LinkId=100392 )
• There must be a second SharePoint Farm installed. This can be a VM somewhere as it only needs to be a single server install. It does however, require that you have enough space presented to the VM in order to restore any site and its database. Consider the size of your largest Content database and its URL structure and add about 20%. This is your starting point for the storage on your recovery server.
• The recovery server will have to have a web application called DPMRecoveryWebApplication created on it. If this name is not found, recovery of some SharePoint data will not be possible.
• The WSS writer must be registered and configured for DPM to use.
• DPM must have use of an account which is a Farm Administrator otherwise, it will not have sufficient permissions.
• The DPM Agent specifically requires the VSS patch 940349 and .NET Framework 2.0 to be installed in addition to the SharePoint requirements.

Let’s start with the following assumptions:

1. SharePointA – Production SharePoint server
• Port 8080 – Administration port
• Port 8081 – Production Website port
• Port 8083 – Alternate Recovery Site on production Farm.
2. SharePointB – Recovery SharePoint server
3. DPMRecoveryWebApplication – web application only created on SharePointB.
4. DPM Agent installed on both SharePoint servers.
5. SharePoint patch 941422 has been installed on both SharePoint servers.

With this configuration, we can restore any site to its original or alternate location or we can also recover a Content Database or the entire Farm.

To start and register the WSS Writer service:

1. Open a command prompt window using elevated privileges. From within the Command Windows, navigate to C:\Program Files\Microsoft Data Protection Manager\DPM\bin and launch 'ConfigureSharepoint.exe'
2. In the command window that appears, you will see the request "Enter the user name for “WSSCmdletsWrapper:" Enter the account name for the user account you granted Farm Administrator credentials
• If you enter an invalid account, you will receive the following:
• Invalid user name.
• Examples of valid user name are "username@domain" and "domain\username".
• Once you add a Fully Qualified account name and hit enter, you are presented with the next line requesting a password.
3. Enter the password for 'WSSCmdletsWrapper':

After a moment of configuring, the entry "The command completed successfully" will appear and the CMD window will close.

DPM only offers the option of protecting the entire SharePoint farm as seen in the screenshot below. This changes when the discussion revolves around recovery of data, as we will see in the next section. DPM provide a great deal of granularity when it comes to recovery of SharePoint data.

If the WSSCmdletsWrapper is not registered properly, then the SharePoint farm you see in the screen shot above will not appear. If it is missing, run the ConfigureSharePoint.exe utility on a SharePoint front-end web server to allow DPM to reconfigure the necessary options.

At this stage, you should be able to create a DPM Protection Group which contains the SharePoint servers take an initial replication of the content. Depending on the size of the sites, this can take a few minutes to hours to complete.

Since there are numerous distributed SharePoint implementations across many servers, we need to talk about which of these servers to protect. Here is a short list.

Web Front-end servers – Since all are the same, only one will need to have the ConfigureSharePoint.exe on it to register the WSS Writer.

Configuration Database Server – The data stored here is unique and not stored on other database servers so it is vital that you have a DPM Agent installed to protect it.

Content Database Servers - Content databases host all the information, content, and data of the farm. Each needs to be protected by a DPM Agent.

Creating the DPMRecoveryWebApplication in SharePoint

Before DPM can be used to recover any data to a protected Farm, the DPMRecoveryWebApplication must be created on the recovery farm’s server. As a requirement of the restoration process, it is helpful to understand the process necessary to create this SharePoint Web Application.

Below are the steps necessary to create the DPMRecoveryWebApplication.

1. Open the SharePoint 3.0 Central Administration console from the Start Menu.
2. Once the SharePoint 3.0 Central Administration console is open, click on the Application Management tab which opens to display all of the various options for managing your SharePoint installation.
3. Look under the SharePoint Web Application Management heading to find the Web application list at the bottom and select this menu item.
4. When the Web Application List appears, check through the list to confirm that the DPMRecoveryWebApplication does not already exist. If it does exist, you are free to delete it and continue with these instructions or you may continue with the one that exists. If it does not exist, please continue through the following steps to create one.
5. Click the “back arrow” to return to the Application Management page. Under the SharePoint Web Application Management heading, click on the Create or Extend Web application.
6. The Create or Extend Web application page appears with two options. The first called Create a new Web application is the one we will choose. The second option, Extend an existing Web application is not needed for this purpose.
7. After clicking the Create a new Web application link, the Create New Web Application page appears displaying various necessary fields for details about the web application that is being created. Here is a brief summary of the important fields of this page.
• Chose the Create a new IIS web site and in the description, name it “DPMRecoveryWebApplication”.
• In the Port field, give it a unique port which is not shared by any other internal application. In the example below, note that the port is 911 since this is our emergency recovery web application for SharePoint failures.
• The Host Header and Path fields can remain unchanged.
• Scroll down to the radio button Create new application pool and in the Application pool name box, note that the entry name includes the port number and the SharePoint recovery server name.
• Under the Select a security account for this application pool, select the Configurable radio button and type in a suitable account and password in the fields that follow. Note the security warning at the top of the page indicating that the credentials of this account may be transmitted in clear-text across the network.
• Lastly, in the Database Name box, type in DPMRecoveryWebApplication.

This is the last entry to make before hitting OK to have the web application created. This will take a few moments but once complete, DPM will be able to recover SharePoint data. Upon completion of the DPMRecoveryWebApplication web application, the Application Created screen appears. At this point, schedule time to run ‘iisreset /noforce’ and once complete, DPM will be able to recover SharePoint data using this recovery farm.

Once these steps have been completed such as the installation of the DPM Agent on the various SharePoint servers, registering the WSS writer, creation of the DPMRecoveryWebApplication on the recovery server, and creation of a recovery site in the production farm, you are ready to protect your SharePoint data.

Next, we will take a look at the steps necessary to recover the entire SharePoint farm whether the production farm is available or not.

How to install Data Protection Manager to a Windows 2008 Server

Article http://technet.microsoft.com/en-us/library/bb808814(TechNet.10).aspx describes how to install DPM on to a 2008 Server. However I have found that this article is incomplete. The following is how to install DPM onto a 2008 Server.

After the 2008 Server setup has finished perform the following:

1. Click Start, point to Administrative Tools, and then click Server Manager.

2. Expand Server Manager to the Features node, and then select Features.

3. In the Features pane, click Add Features.

4. Select Windows PowerShell, and then click Next.

5. On the Confirm Installation Selections page, click Install.

6. Click Start, point to Administrative Tools, and then click Server Manager.

7. Expand Server Manager to the Roles node, and then select Roles.

8. In the Roles pane, click Add Roles.

9.  In the Add Roles Wizard, on the Before You Begin page, click Next.

10. On the Select Server Roles page, select Web Service (IIS).

11. In the Add features required for Web Server (IIS)? message box, click Add Required Features.

1. Ensure that you select the following Role service:
2. HTTP Redirection
3. Application Development
   1. ASP.net
   2. .NET Extensibility
   3. ISAPI Extensions
   4. ISAPI Filters
   5. Server Side Includes
4. IIS 6 Management Compatibility
   1. IIS 6 Metabase Compatibility
   2. IIS 6 WMI Compatibility
   3. IIS 6 Scripting Tools
   4. IIS 6 Management Console
5. Security (Installed)
   1. Windows Authentication (Installed)

Note: if you do not install these components correctly you will get an error during DPM installation.

12.  Install SIS

1. At a command prompt (with elevated privileges) type:
   1. Ocsetup.exe SIS-Limited
   2. Press enter
   3. Wait for the command to complete (this can take a while)
   4. Reboot the box.
      1. To ensure that the SIS service gets installed you can check the following key. This key will appear after a reboot.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SIS

13.  Run the DPM installation wizard.

14, Install hotfix 950082 Description of the Data Protection Manager 2007 hotfix package rollup 2

http://support.microsoft.com/default.aspx?scid=kb;EN-US;950082

Author:  Keith Hill
Microsoft Enterprise Services

How to optimize your Windows Server 2003 Print Failover Cluster

There are a lot of administrators that need to maintain a highly available Print Server running on Windows Server 2003 RTM/SP1/R2/SP2 Failover Clusters

Hopefully with this series of blog posts, you will be able to create a solid printing environment where you can maintain close to 99.99% uptime* and have good performance during a move or failover.

I'm going to be assuming that you have already have either built a brand new Windows Server 2003 Print Failover Cluster or are migrating from a Windows 2000 Server Print Failover Cluster.**

Personally, I prefer that you build a brand new O.S. instead of migrating, this provides you to start fresh.  All you have to do is migrate the actual print queues.

After you have built it, I hope that you have it optimized per:

• 895092 Recommended hotfixes for Windows Server 2003-based server clusters
• 923830 Recommended hotfixes for Windows Server 2003 Service Pack 1- based server clusters
• 935640 Recommended hotfixes for Windows Server 2003 Service Pack 2-based server clusters

Like a house, you want to have a strong structural foundation before adding items.

There are some whitepapers such as "Creating Highly Available Printers with Windows 2003 Server Cluster"  in http://www.Microsoft.com/PrintServer where all Administrators should start.

When you are done reading that whitepaper, the question becomes so what do you need to optimize and make your print cluster server scalable?

1. What is your Cluster reset quorum log set to?
2. Do you need Terminal Services Print redirection turned on?
3. Do you need local print drivers on each physical node?
4. Do you have the print drivers that you actually use and need?
5. Do you have the latest versions of the OEM print drivers?
6. Do you have the hotfixes needed for printing?

In future posts, we will be discussing how to go about troubleshooting these 6 items. We'll also cover Windows Server 2003 to Windows Server 2008 Print Failover Cluster migrations and best practices.

Author: Yong Rhee
Support Escalation Engineer
Microsoft Enterprise Platforms Support

* When using inbox print drivers (Print drivers that ship with Windows Server 2003)
** Guide to Creating and Configuring a Server Cluster under Windows Server 2003 White Paper

Data Protection Manager 2007 – Important information for using ESEUTIL when configuring Exchange Server Protection.

In Data Protection Manager 2007, when configuring a protection group for protection of Exchange data for the versions that DPM supports (Exchange Server 2003 w/SP2 or higher and 2007), you are given an option to enable execution of ESEUTIL.EXE against those Exchange data sources you wish to protect. When enabled, DPM performs ESEUTIL consistency checks on the data written to disk or tape to ensure that the backup is not corrupt and that it will be readable when or if it’s required. Enabling this also minimizes backup-related performance impact by offloading the typical ESEUTIL operations that most backup applications force on the production server and instead the DPM server handles this on the redundant data, lightening the load on the production Exchange Server so that it can serve more users with the goal to have less of an impact on production.

When the ESEUTIL option is enabled it’s important to take careful consideration concerning the architecture (x86, x64) of both the DPM and Exchange Server. For example, if the DPM 2007 server is 32-bit and you’re protecting a 64-bit 2007 Exchange Server, you MUST use the 32-bit ESE.DLL and ESEUTIL.EXE file versions.

Another important item to note concerning 64-bit Exchange 2007 Servers revolves around those upgraded to Exchange SP1. In a situation as mentioned above where the DPM Server is 32-bit, in order for you to retain the ESEUTIL option and leave it enabled, you must update the two ESE files on your DPM server to the SP1 versions. This can be accomplished using one of the options below:

1. Copy the files from a 32-bit Exchange 2007 Server that is on Service Pack 1 for Exchange.

2. Install the 32-bit Microsoft Exchange Server 2007 Management Tools on a machine of your choice then upgrade the tools by installing the 32-bit Exchange Server 2007 SP1. You can then simply copy the eseutil.exe and ese.dll files from the “%programfiles%\Exchsrvr\bin” folder on the machine to be placed on the DPM server.

Once you’ve obtained a copy of the correct versions, you’ll need to manually copy these into the following folder on the DPM server:

“%programfiles%\Microsoft DPM\DPM\bin”

Well, you ask,  'What if I’m protecting both 2003 and 2007 versions of Exchange Server?"   The good news here is that you only need to gather the files from the protected Exchange Server running on the latest build as the files will retroactively work with the older versions of Exchange running in the environment. In other words, copy the files from the 2007 Exchange Server onto the DPM server.

Below is an example of the prompt Data Protection Manager will display if you try to enable the ESEUTIL option without having manually copied the files from Exchange server onto the DPM Server:

Examples of DPM 2007 failed job alerts that you are likely to encounter if you’ve copied the wrong and incompatible versions onto your DPM server:

Type: Recovery point
Status: Failed
Description: Data consistency verification check failed for LOGS of Storage group
First Storage group on servername.contoso.com. (ID 30146 Details: The image file %1 is
valid, but is for a machine type other than the current machine (0x800700D8))
End time: 2/27/2008 7:04:54 AM
Start time: 2/27/2008 7:03:55 AM
Time elapsed: 00:00:58
Data transferred: 7.13 MB
Cluster node -
Recovery Point Type Express Full
Source details: First Storage group
Protection group: Exchange Server Protection

Affected area: First Storage Group
Occurred since: 2/27/2008 7:16:25 PM
Description: The replica of Storage group First Storage Group on servername.contoso.com is
inconsistent with the protected data source. All protection activities for data
source will fail until the replica is synchronized with consistency check. You can
recover data from existing recovery points, but new recovery points cannot be
created until the replica is consistent (ID 3106)
Data consistency verification check failed for LOGS of Storage group First Storage
Group on servername.contoso.com. (ID 30146 Details: Unknown error (0xfffffdfe)
(0xFFFFFDFE))
Recommended action: Either the database files are corrupt or the proper versions of
the following files are missing.
If you have recently upgraded your Exchange server, please copy them from that
server to DPM server.
eseutil.exe
ese.dll
Please contact the administrator for Exchange server, and verify the issue. You can
recover the last known good backup to address the corrupted state.
Synchronize with consistency check.
Run a synchronization job with consistency check...
Resolution: To dismiss the alert, click below
Inactivate alert0xfffffdfe

Author:Tyler Franke
Support Engineer
Microsoft Enterprise Platforms Support

Protecting multiple Domain Controllers in a multi-DPM server environment

If you are protecting more than one domain controller, and using more than one DPM server to do it, you may experience an issue where at random times, all servers being protected by Data Protection Manager ("DPM") may lose connection with the DPM server. They may lose connection over a short period of time, not all at once. This can occur with both Data Protection Manager 2006 and 2007, or a mixture of both.

This can occur if the domain controllers are protected by more than a single DPM server. In other words, one domain controller is protected by one DPM server, and another domain controller is protected by a different DPM server.

Another symptom may be the existence of bogus entries in Users container in Active Directory Users and Computers. These entries will be similar to DPMRADCOMTrustedMachinesCNF:28f84c90-fa10-4ff7-b4fa-7d945440e08b.

This issue is caused by the replication process between the domain controllers.

There are two workarounds for this issue:

1. Have all domain controllers protected by a single DPM server.

2. To allow domain controllers to be protected by different DPM servers, do the following:

Step 1. Delete all the bogus DPMRADCOMTrustedMachinesCNF:{GUID} on ONE domain controller, then wait a few hours for the changes to be replicated throughout your environment including remote sites. The amount of time required will vary based on the domain topology.

Step 2. On ONE domain controller, confirm that all DPM server names are members of the following groups:

<Domain>\Users

DPMRADCOMTrustedMachines

DPMADmTrustedMachines

Builtin

Distributed COM Users

Allow sufficient time for the changes to replicate throughout the domain.

Author: Kevin McNiel
Support Engineer
Microsoft Enterprise Platforms Support

Indiana Jones and the search for Hyper-V Information

In the last episode, Indy was hanging precariously from a ........
No seriously..... a high-demand commodity right now is information about Microsoft's new virtualization technology - Hyper-V.  There is actually a good bit of information floating around cyberspace but, unfortunately, not all of it is centrally located and it may be difficult to find. To help with that search, here is a listing of some of the best locations to look for the latest and greatest information that is publicly available:

Websites

Microsoft Virtualization website

Microsoft TechNet Virtualization Center

Windows Server 2008 Server Virtualization TechNet Forum

Hyper-V Performance Counters

Hyper-V WMI Provider (MSDN)

Blogs

Microsoft Virtualization Team Blog:

Virtual PC Guy

John Howard's Blog

Robert Larson's Blog

Rakesh's Virtual Management Blog

Virtual Varia

Jose Barreto's Blog

Admittedly, there are many, many blogs out there containing scraps of Hyper-V information.  The one's listed above are the main ones and will serve as great jumping off points. Let us know your favorite Hyper-V sources.  Good hunting!

Author: Chuck Timon
Support Escalation Engineer
Microsoft Enterprise Platforms Support

Data Protection Manager - Tape drive inventory is not showing up properly

If the information under the Libraries tab in the Management Task area does not show up correctly:

Follow the steps below to remap the tape drive information.

• Stop the DPMLA service if it is running
• Remove any tapes from the drive and insert at least one non cleaner tape in the library
• Create a backup of the following file if it is available: %ProgramFiles%\Microsoft DPM\DPM\Config\dpmla.xml
• Run DPMDriveMappingTool.exe from the %ProgramFiles%\Microsoft DPM\DPM\Bin directory
• Start the DPM Administrator Console the rescan the libraries from the Management/Libraries tab

Author:Patrick Lewis
Support Escalation Engineer
Microsoft Enterprise Platforms Support

Multi-Site Failover Cluster Communications Connectivity

The networking model changed significantly in Windows Server 2008 Failover Clustering. First, and foremost, when we start talking about networking in 2008 Failover Clusters we need to drop some of the older concepts that have been around for awhile. Probably the one that needs to go first is the concept of 'Private' network. This term no longer applies in W2K8. All networks detected by the cluster service, and hence the cluster network driver, are, by default, classified as 'mixed' networks and are automatically configured for use by the cluster. Here is a sample of the default configuration for a network in a cluster -

Whether the cluster gets to use this network or not for cluster related communications (e.g. health checking, GUM updates, etc....) is determined by the check box next to "Allow the cluster to use this network.". You will also note that the box is checked for "Allow clients to connect through this network." This is the default configuration provided the network interface supporting the network has a default gateway configured. So, no DG configured....no client access is allowed unless a cluster administrator changes the default setting in the Failover Cluster Management interface. Inspecting the cluster registry settings for this network, the 'Role' is set = 3 which is a 'Mixed' network. So, cluster either gets to use the network or it does not......no more 'Private' network.

In Windows Server 2008 Failover Clustering, the cluster network driver underwent a complete re-write because it had to support new features including allowing cluster nodes to be placed on separate, routed networks. The new driver is 'netft.sys' (Network Fault-Tolerant driver). When loaded, it shows up as a network adapter (Microsoft Failover Cluster Virtual Adapter) and when you run an 'ipconfig /all' it is listed in the output with a MAC address and an AutoNet address (no modifications are needed nor desired). As a 'fault-tolerant' driver, one of its main functions is to determine all the network paths to all nodes in the cluster. As part of this process, netft.sys builds its own, internal routing table to find each node in the cluster on port 3343 -

Hence, the requirement for a minimum of two network interfaces or you get a Warning message when running the networking validation tests...a single NIC cluster node violates a 'best practice', that being a single point of failure for cluster communications...and therefore, is not a supported configuration. Cluster network communications have changed from being UDP Broadcast to being UDP Unicast to accommodate the routing required since most routers, by default, will not route broadcast traffic. Plus, Unicast is more efficient. Here is an example packet in a network trace -

But wait.....UDP communications are not reliable! Not true! Provided the network these packets are traveling on is of good quality (not excessive packet loss), UDP communications are fine. In fact, they are more efficient since they do not have the overhead of TCP where it has to wait around for ACKs, and if not received, all the retransmission's that clog the network. Since there are no acknowledgements associated with UDP traffic, the cluster has some built-in 'enhancements' that allow for tracking of the messages that are sent to all nodes in the cluster. There is even a capability for retransmitting messages if they are not acknowledged by a cluster node (but this topic is beyond the scope of this thread)...suffice it to say, there is built-in reliability in cluster communications.

Not all cluster communications use UDP-Unicast. There are some communications that use TCP. For example, during the 'join' process, all initial communications use TCP. Once a node has successfully joined a cluster, intra-cluster communications can then include UDP as needed.

The cluster log also provides valuable information about how the cluster service builds its 'networking knowledge' as part of its startup routine. In the cluster log, you can follow as the cluster architecture components (Interface Manager (IM) and Topology Manager (TM) for example) validate communications connectivity between nodes in the cluster. Some of the information eventually becomes part of the cluster network driver routing table. Here is an example -

So, if you want to allow client access on a network in a cluster by default, it will have to have a default gateway assigned to it, or you will have to manually configure the network for client access.  If you do not have multiple networks that can be accessed via multiple routes, then the cluster will have a difficult time building a routing table that has enough information to provide reliable, cluster-wide communications over multiple networks.

There are other considerations when configuring multi-site clusters.  Start by reviewing this - http://support.microsoft.com/kb/947048/en-us

Author: Chuck Timon
Support Escalation Engineer
Microsoft Enterprise Platforms Support

Data Protection Manager: What is a 'Consistency Check' and what could cause it to fail?