How to Use The Migratedatasourcedatafromdpm.Ps1 DPM Powershell Script to Move Data

How to Use The Migratedatasourcedatafromdpm.Ps1 DPM Powershell Script to Move Data

  • Comments 17
  • Likes

The Migratedatasourcedatafromdpm.Ps1 DPM Powershell Script  is Included in Service Pack 1 of Data Protection Manager 2007.

The MigrateDatasourceDataFromDPM is a command-line script that lets you migrate DPM data for individual “data source(s)” or all Replica volumes and recovery point volumes to different physical disks. Such a migration might be necessary when your disk is full and cannot be expanded, your disk is due for replacement, or disk errors show up.

Depending on how you have configured your environment, this could mean one of more of the following scenarios for moving data source data:

· DPM Physical disk to another DPM Physical disk

· DPM Data source to different DPM Physical disk

· DPM Data source to Custom volume.

The MigrateDatasourceDataFromDPM script moves all data for a data source or disk to the new volume or physical disk. After migration is complete, the original disk from where the data was migrated from is not chosen for hosting any NEW backups, however the recovery points located on the source disk can be used for restores until the recovery points are expired.

Note: You must retain your old disks until all recovery points on them expire. After the recovery points expire, DPM automatically de-allocates the replicas and recovery point volumes on these disks.

All backup schedules continue to apply and protection of the data source continues as before, but will use the new disk.

After migrating the replica of a data source that has secondary protection enabled, you must start the Modify Protection Group wizard on the secondary DPM server, select the same data source, and complete the wizard. This reconfigures secondary backups to run from the new replica volume on the primary DPM server.

I will walk you through the steps on migrating data source (disk and data) to help you understand what the required commands and the results once the command has completed successfully.

In this first scenario we are going to use the MigrateDatasourceDataFromDPM to conduct a DPM disk to DPM disk migration from start to finish.

In the example below you can see in Disk Manager Disk 1 and Disk 2 is utilized for the DPM storage pool and the replica and recovery volumes are spread across both disks.

clip_image002

From within the DPM UI Protection Group Tab you will see that we have four protection groups with a number of different data sources (Share, SQL, Volume, etc.)

clip_image004

Within the DPM UI Management Tab under Disks you see that we have Disk 1 and Disk 2 allocated to the DPM storage pool

clip_image006

Now we have added two new physical disks to the DPM server which is running Data Protection Manager 2007 SP1, as you will note Disk 3 (4.88GB) and Disk 4 (146.48GB) are listed in Disk Manager and are unallocated and currently basic disks.

clip_image008

After walking through the process of adding Disk 4 as an additional disk to the DPM Storage Pool, you will see that it is now listed in the DPM UI and shows up as 100% unallocated space.

Adding Disks to the Storage Pool

http://technet.microsoft.com/en-us/library/bb795901.aspx

clip_image010

We will now open the DPM command shell and run a command (Get-DPMDisk -DPMServerName <DPM Server Name>) to display the disks.

Get-DPMDisk -DPMServerName RKW2K3-DPM

In order to use the migration powershell command you must use a variable name to hold the array of retured items. In the example below, we have used the variable $disk to hold the Get-DPMDisk -DPMServerName <DPM Server Name> output.

$disk = Get-DPMDisk -DPMServerName RKW2K3-DPM

After running the command you will notice that there are four disks listed, and they are not necessarily arranged in order that disk management lists them. Note that the NTDiskID is the physical disk number (zero based) that disk management lists in the GUI. Note that the NtDiskID are not in numeric order and that disk 0 (windows operating system disk) is not included in the output.

clip_image012

We are now going to use the MigrateDatasourceDataFromDPM.ps1 script to migrate the DPM Physical Disk 1 to Physical Disk 4. ( $disk array element [2] to array element [1] )

(./MigrateDatasourceDataFromDPM.ps1 -DPMServerName <DPM Server Name> -Source $disk[n] -Destination $disk[n])

When using this command the $disk[number] that is used within the brackets is not the NTDiskId but the is the element number in the array list in the $disk variable. This number is always zero based, meaning the 1st element in $disk[0] is physical disk 3 in the above screenshot.

Looking at the output when running the command $disk “DPM Physical Disk 1 is third element in the list starting with 0 this will make Physical Disk 1 = [2] in the list and Physical Disk 4 = [1] in the list so our command will be as follows;

./MigrateDatasourceDataFromDPM.ps1 -DPMServerName RKW2K3-DPM -Source $disk[2] -Destination $disk[1]

clip_image013

The command may take some time depending on the number and size of the volumes on the source disk and once completed you will be back at the DPM Shell prompt.

clip_image015

You will now notice in Disk Management the DPM replica and recovery point volume information which is location on Disk 1 and Disk 2 has been migrated to Disk 4. Any new recovery points for the respective data source will now be located on the new volumes on the new disk, the original volume data on Disk 1 and Disk 2 will still need to be maintained until the recovery point on them expire. Once all recovery points expire on the old disk(s), they will appear as all unallocated free space in disk management, and can then be removed from Windows or be reused.

The MigrateDatasourceDataFromDPM script moves all data for a data source or disk to the new disk or volume. After migration is complete, the original disk from where the data was migrated is not chosen for hosting any new backups. You must retain your old disks until all recovery points on them expire. After the recovery points expire, DPM automatically de-allocates the replicas and recovery point volumes on these disks.

clip_image017

Also since we did a disk migration of Disk 1 to Disk 4, Disk 1 no longer shows up in the DPM UI and will not be used any further for DPM Storage Pool this is normal and is as expected.

clip_image019

After completing the disk to disk migration you will also notice that all of the Protection Groups which used Physical Disk 1 for either or both volumes (replica and Recovery Point) will now show up in DPM as Replica is inconsistent. This is normal and is expected as there has been changes made to the volume and will need to be re-synchronized by running a synchronization job with consistency.

clip_image021

After we have completed the Synchronization job with consistency, all of the Protection groups are now all consistent and up to date and have a Protection Status of OK.

That concludes the Disk to Disk migration, in my next blog we will walk through the process of conducting a Data Source to Disk migration and see how this will help in minimizing the amount of volumes a data source uses.

 

 

Author:
Robert Kierzek
Senior Support Engineer
Microsoft Corporation

 

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • while using the array index on $disk works, it relies on you correctly identifying the disk in the array. Tidier to use $source = get-dpmdisk | where {$_.ntdiskid -eq 2}. Similarly, I would then set $dest = get-dpmdisk | where {3,4 -contains $_.ntdiskid} (note that I can specify all the disks in my new storage pool). Migration command would then be ./MigrateDatasourceDataFromDPM.ps1 -DPMServerName RKW2K3-DPM -Source $source -Destination $dest

  • Hello,

    when are you going to publish your next blog about minimizing the amount of volumes a data source uses. I have to reduce the size of the volumes in DPM. I backup Exchange and half of the volume of DPM are empty. I want to move this volume and minimize the sie of the volume. How can I do it?

    regards

    Blaise

  • Can this be used to move data onto a volume that is already in use?

  • Same question as Mike, we have 2 volumes we want to move to one volume, can this be done with this method?

    Cheers.

  • Can this be used to move data onto a volume that is already in use? I know this has been asked before.

  • How do you migrate from one protection group to another or can this not be done?

  • Does anyone know the answer to the question that has already been asked?  I need help too.  

    "Can this be used to move data onto a volume that is already in use?"

  • When are you going to post the next blog....."in my next blog we will walk through the process of conducting a Data Source to Disk migration" ????????????????????????

  • Yeah...I want to know how to do Data Source to Disk migration, too!!!  Please post the next blog!!

  • I also want to move data from 1 storage pool disk to another storage pool disk, both of which currently h ave DPM volumes on them.

  • Hi all, I happened to come across the blog for a refresher on migrations and found several unanswered questions.  I'll try to help.

    @ simonw:  Thanks for the tip!  I'm going to try using $source this time.

    @ all asking about migrating to a disk already containing data:  

    Yes, this script can be used to migrate to volume(s) containing data.  If you have several destination disks, you can put them in a variable such as $destination = $disk[1], $disk[5], $disk[8], etc and the script will fit the source data onto the destination volumes.

    @ all asking for the next blog entry:  

    I'm assuming most of you are interested in consolidating a large datasource's replica and/or recovery point volume.  As a datasource grows over time, its replica and/or recov point volume can have extents across several disks in the DPM storage pool.  See technet article here: technet.microsoft.com/.../dd282970.aspx and look at example #3, which is a very simplified version of what the next blog article would be.  If you apply the lessons from *this* blog entry, you can accomplish the task.  Now to describe example #3 more towards what *I think* you're asking, let's say D1 contains R1 & RP1.  But R1 has grown several times so it has extents on D2, D4, D5, & D7.  You can run the script to a destination, let's say D8, and it will consolidate all those extents to a single R1' on D8.

    @ BVille32:  This script cannot migrate between protection groups.  You would need to remove a datasource (be sure to check the box to keep recovery points) and then re-add the datasource to the other protection group.

    @ Blaise2004:  This script cannot reduce the size of the volume but there are two potential methods to reduce the sizes.  #1)  DPM 2010 has a 'LUN shrink' option.  #2)  If you have multiple DPM servers, you could remove protection from one DPM (keep recovery points) and use the sizing values to manually enter the correct sizes when you add the datasource back to the second DPM server.

  • hello robert

    I have DPM 2010 and hundred of simple volumes, it is correct to see on disk management all this volumes? I almost see a line that actually is a volume, it is a bad practice to have for example, 300 simple volumes showing at disk management? (diskmgmt.msc)

    thank you!

  • Dear Robert,

    I my senario I have migrated all protection groups to the new disk pool but I could not retain old disk till the retention range over. now I am getting LDM database occupancy error when i try to create new protection group

  • Such a migration might be necessary when your disk is full and cannot be expanded, your disk is due for replacement, or disk errors show up.  <a href="http://www.rattan-lounge-moebel.ch">lounge gartenmöbel</a>

  • Hi,

    DPM 2012

    I have migrated data from disk to disk , this seemed to go without any error but the original disk is still showing the same disk size and indicates protected data sources on disk.  I have run the PruneShadowCopies2010.ps1 script but this does not seem to do anything.  I have also run a modified script to delete all shadow copies older than 14 day but still the disk is not showing the free space I presumed it would.  This issue is I want to remove the disk from DPM but I am worried now as It is still displaying protected data sources.  Any ideas???

    Thanks

    Richard