Unable to Extend System Volume & DPM Backup Failure: Change the System State backup location

I recently had a couple of servers where they had rather small system partitions, one was Windows 2003 while the other was Windows 2008 RTM and DPM continued to fail with replica is inconsistent errors.  The DPM documentation and event errors were referencing the replica and volume partition size(s) and all checked out with plenty of space according to the current data size. 

Fix #1:  Expand the available space for the system partition

As a test, I thought about checking the actual system partition size on the protected servers and noticed small free space available on these virtual machines.

For testing purposes, I shut the virtual machine and expanded the system partition virtual hard disk.  To do this, I simply did the following:

1. Shut the virtual machine down in the VMM Administrator Console
2. Right-click and selected properties on the virtual machine
3. Highlighted IDE disk 1 and checked compact and expand disk and selected the expanded space size
4. Click OK
5. Restarted the virtual machine upon the job completion
6. Opened Server Manager (Server 2008 only), elected disk storage, and right-clicked and chose Re-scan disks
7. Right-clicked the system partition and selected to Extend volume

After the volume free space is sufficient (greater than 30+ gigabytes was my goal), then I re-run the consistency check for the replica partition for the server(s) that failed.

To re-check the consistency check, you do the following:

1. Open the DPM Administrator console
2. Click the Protection Tab
3. Locate the protection group, and expand if not already expanded
4. Locate the protected server and expand it to see the current volumes and system state backup
5. Right-click on the failed state item and select “Perform Consistency Check”

Fix #2:  Re-locate the location for the System State backup location on the protected server

The one problem that many might run into is the system partition isn’t expandable using the “extend volume” as it is either not supported (Windows Server 2003) or is a physical disk limitation.  In either case, the choice at this point is less related to creating the relevant space in order to successfully get system state backups to succeed and more pointed at moving the system state backup to another location. 

NOTE:  If you have limited space for your system partition and no other volume with significant space available to host the backup, you have bigger problems than the replica backup failing.  You should fix the limited free space first for this server and then focus on the following fix.

As I wasn’t familiar with how to successfully re-locate system state backup location, I used Bing search to see if others had figured this out already.  I came across this small blog post that was very helpful and for the most part was completely accurate thought I did have to make one minor tweak to finally get things happy (e.g. no failures or errors, just the big old Green OK check mark).

Follow these steps outlined in the post above -

1. Login to the server with credentials that is an Administrator
2. Navigate to the installation directory for the DPM agent, typically %programfiles%\dpm
3. Double-click datasources directory
4. Right-click on the PSDataSourceConfig.xml file and select open with and choose Notepad (or your preferred text editor)
5. Locate the <FilesToProtect> element
6. Change the current path from %systemdrive%\WindowsImageBackup\* (if Server 2008) or %SystemDrive%\DPM_SYSTEM_STATE\%computername%.bkf (if Server 2003) to the location you would like it to reside in (for example, H:\WindowsImageBackup)

As mentioned in the blog post above, DPM’s protection group will now go south on you and it will seem permament.  This is expected since the location of the system state configuration has changed.  The protection group needs to be modified through the wizard and as mentioned it should do so with no changes.  This, unfortunately, didn’t do the case every time and some protection groups were still failing.  This was easily determined using the DPM Events in the event viewer for the server with the following:

“DPM has detected changes in the file locations or volume configurations of protected objects…”

In order to correct this failure, I had to use the modify protection group wizard and remove the System State from the protected server and add it back.  After doing this, the replica consistency started working as planned.

Summary

In the short time I’ve started using DPM 2007, I’ve found it to be rather straight forward to troubleshoot failures with replica backups.  The issue is usually related to volume space not large enough, in my experience thus far.  With some quick cleanup, you can easily turn things around assuming you have the disk space (got to love SAN’s and DAS!) and the DPM console.

Hope this helps,

-Chris