toolsignThere are many failures that can happen when protecting Hyper-V hosts and CSV’s that are a result of the Cluster network configuration not being optimized for redirected mode communication. In most cases these problems will only happen under a heavy load, and when it happens we can lose access to the storage that houses the CSV, Virtual machines may reboot or backups may simply fail.

The following errors codes will be logged on the DPM server when the Cluster network configuration is not optimized. These error codes assume that you have already installed Service Pack 1 for Windows Server 2008 R2. Install it to get the available VSS and Hyper-V component updates.

ErrorCode: 0x8007173D (-2147018947):
Back up is in progress. Please wait for backup completion before trying this operation again.

ErrorCode: 0xC000020C (-1073741300):
The specified network name is no longer available.

ErrorCode: 0x80042308 (-2147212536):
VSS_E_OBJECT_NOT_FOUND: The specified object was not found.

ErrorCode: 0x8099090E (-2137454322):
CMDPROC_E_RESPONSE_TIMEOUT (cmdtypes.h:122)

ErrorCode: 0x8007274C (-2147014836):
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

The first step in addressing these errors is to use the following KB article to verify that the Cluster network configuration is configured as expected:

2473194: DPM 2010 Hyper-V protection Configuring cluster networks for CSV redirected access (http://support.microsoft.com/default.aspx?scid=kb;EN-US;2473194)

Attempting to utilize a single network adapter for all traffic will quickly expose these problems. More commonly, a separate Live Migration network will not be setup and used, thus the network will quickly become saturated.

There are also post SP1 hotfixes available that will resolve many of the network communication problems:

2263829: The network connection of a running Hyper-V virtual machine may be lost under heavy outgoing network traffic on a computer that is running Windows Server 2008 R2 SP1 (http://support.microsoft.com/kb/2263829)

961804: Virtual machines are missing in the Hyper-V Manager Console or when you create or start a virtual machine, you receive one of the following error codes: "0x800704C8", "0x80070037" or "0x800703E3" (http://support.microsoft.com/default.aspx?scid=kb;EN-US;961804)

2494016: Stop error 0x0000007a occurs on a virtual machine that is running on a Windows Server 2008 R2-based failover cluster with a cluster shared volume, and the state of the CSV is switched to redirected access (http://support.microsoft.com/default.aspx?scid=kb;EN-US;2494016)

2494162: The Cluster service stops unexpectedly on a Windows Server 2008 R2 failover cluster node when you perform multiple backup operations in parallel on a cluster shared volume (http://support.microsoft.com/default.aspx?scid=kb;EN-US;2494162)

When all of these changes have been implemented you can also add the following registry keys to increase the number of retries and decrease the time between retries.

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Data Protection Manager\Agent\CSV]
"CsvMaxRetryAttempt"=dword:000000C8
"CsvAttemptWaitTime"=dword:0002bf20

CsvMaxRetryAttempt: Adjusts the maximum number of times (Default is 1) the DPM agent will attempt to claim the CSV volume. The value 0xC8 = 200 times.

CsvAttemptWaitTime: Adjusts the amount of time in milliseconds to wait between retry attempts. The value 0x2bf20 = 3 minutes.

Michael Vargo | Support Escalation Engineer

App-V Team blog: http://blogs.technet.com/appv/
AVIcode Team blog: http://blogs.technet.com/b/avicode
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
OOB Support Team blog: http://blogs.technet.com/oob/
Opalis Team blog: http://blogs.technet.com/opalis
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
OpsMgr Support Team blog: http://blogs.technet.com/operationsmgr/
SCMDM Support Team blog: http://blogs.technet.com/mdm/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

clip_image001 clip_image002