Ctrl P - The Data Protection Manager Blog! : How does it work

Enabling/Disabling co-location of data on Tape

dpm — Sun, 06 Jul 2008 20:02:00 GMT

With DPM 2007 Feature Pack (KB949779), we have added support for media co-location. With this feature, you can achieve better tape utilization by co-locating data from multiple protection groups with similar retention range. This feature is disabled by default. You can use following PowerShell commands to enable and disable tape co-location.

Enable Tape co-location:

Set-DPMGlobalProperty -DpmServer <DPM Server Name> -OptimizeTapeUsage $true

Disable Tape co-location:

Set-DPMGlobalProperty -DpmServer <DPM Server Name> -OptimizeTapeUsage $false

You can verify the status of tape co-location feature (Enabled/Disabled) from Library Management tab of DPMUI.

The Feature Pack can be downloaded from following locations.

x86: http://www.microsoft.com/downloads/details.aspx?FamilyID=e9e1fe35-b175-40a8-8378-2f306ccc9e28&DisplayLang=en

x64: http://www.microsoft.com/downloads/details.aspx?FamilyID=ad5cd1a2-9b87-4a2c-90a2-9dbaf1024310&DisplayLang=en

Madhan S

DPM CLI Tips 'n Tricks: Powershell Basics

dpm — Wed, 18 Jun 2008 20:47:00 GMT

The scripts posted in this blog require knowledge of Powershell and DPM cmdlets. So we thought we would present some tips ‘n tricks to become power users!

Introduction

The first difference between any normal Command Line Interface (CLI) (typically, Windows cmd.exe based environment) & Powershell is that, Powershell is a full blown .Net environment and is object oriented. In other words, it treats all input/output parameters as objects and when you pipe one command to another, objects flow instead of plain text as in old shells. Similarly, DPM Management Shell also takes inputs in the form of objects – for example, DPM, Protection Group, Datasource, Library, Tape, Tape Drive, Disk etc. are all .Net objects. Let’s now move on to more interesting stuff.

Finding cmdLets (Get-Command or gcm)

For knowing all the cmdLets present in the Powershell instance, use Get-Command (or gcm in short). When run, the output will be something like this:

PS D:\ > Get-command

CommandType Name Definition

----------- ---- ----------

Cmdlet Add-Content Add-Content [-Path] <String[...

Cmdlet Add-DPMDisk Add-DPMDisk [-DPMDisk] <Disk...

Cmdlet Add-History Add-History [[-InputObject] ...

Cmdlet Add-Member Add-Member [-MemberType]

Cmdlet Add-PSSnapin Add-PSSnapin [-Name] <String...

Cmdlet Add-Tape Add-Tape [-DPMLibrary] <Libr...

Cmdlet Clear-Content Clear-Content [-Path] <Strin...

Finding DPM cmdLets (Get-DPMCommand)

Similarly for getting all the cmdLets belonging to only DPM, use the Get-DPMCommand.

PS D:\ > Get-DPMCommand

CommandType Name Definition

----------- ---- ----------

Cmdlet Add-DPMDisk Add-DPMDisk [-DPMDisk] <Disk...

Cmdlet Add-Tape Add-Tape [-DPMLibrary] <Libr...

Cmdlet Connect-DPMServer Connect-DPMServer [-DPMServe...

Cmdlet Disable-DPMLibrary Disable-DPMLibrary [-DPMLibr...

Cmdlet Disable-TapeDrive Disable-TapeDrive [-TapeDriv...

How to use a cmdlet?

1. Understanding the cmdlet parameters (Get-Command and Format-List)

There are two parts to understanding a cmdlet. First, to look at the various input parameters and various usages, you can use the Get-Command itself on a specific cmdlet, in the following fashion – Get-Command <cmdlet> | format-list (or gcm <cmdlet> | fl, in short).

PS C:\> gcm Set-Alias | fl

Name : Set-Alias

CommandType : Cmdlet

Definition : Set-Alias [-Name] <String> [-Value] <String> [-Description <String>] [-Option <ScopedItemOptions>] [-PassThru] [-Scope <String>] [-Force] [-Verbose] [-Debug] [-ErrorAction <ActionPreference>] [-ErrorVariable <String>] [-OutVariable <String>] [-OutBuffer <Int32>] [-WhatIf] [-Confirm]

2. Reading Help documentation for a cmdlet (Get-Help or help)

For reading the Help documentation for any cmdlet use the following (Get-Help or help)

Get-Help <cmdlet>

For example:

PS D:\ > Get-Help Add-Tape

NAME

Add-Tape

SYNOPSIS

Adds a tape to a DPM library.

SYNTAX

Add-Tape [-DPMLibrary] <Library> [-Async] [-JobStateChangedEventHandler <Jo

bStateChangedEventHandler>] [<CommonParameters>]

…..

3. Getting detailed help, and seeing sample scripts

For seeing additional information on each of the cmdLets, you can use the –Full or –Detailed parameters in Get-Help.

e.g. Get-Help Add-Tape –Detailed

Get-Help Add-Tape –Full

DPM Object Properties (Get-Member or gm)

The DPM cmdlets are logically divided in three groups Protection (Backup), Recovery and Management related (Library & Disk). All the tasks that can be done from the DPM UI in these areas, can be done from the cmdLets in these areas. Infact, the CLI provides additional features than the UI, in some scenarios.

You can get the member properties of any object by pipelining the output to Get-Member:

For example:

$lib = Get-DPMLibrary -DPMServerName “Testing Server Name”

$lib | Get-Member

Will give all the members of $lib (Library object).

PS D:\ > $lib | get-member

TypeName: Microsoft.Internal.EnterpriseStorage.Dls.UI.ObjectModel.LibraryMan

agement.Library

Name MemberType Definition

---- ---------- ---------

ClearCache Method System.Void ClearCache()

Dispose Method System.Void Dispose()

We hope this was useful, feel free to add comments as feedback to this post. And in the next version, we can add more tricks!

- Mukul Shekawat, Balaji Hariharan

Why does DPM use the Recovery Storage Group in Exchange?

JasonBuffington — Thu, 05 Jun 2008 17:27:46 GMT

One of the more common questions that we hear from DPM customers is around the DPM dependency on the Exchange RSG, while other backup technologies do not require it.

How Microsoft customers were protecting Exchange previously

When Microsoft looked at how customers were protecting Windows applications today, we found two troubling scenarios:

1. Most customers were using one or more technologies for traditional nightly tape backup, particularly within heterogeneous environments. In addition, many were using or considering the use of different technologies for disk-to-disk replication, often per workload. And some were also using a third type of storage technology for long-distance synchronization for disaster recovery or business continuity solutions. Often the mix of these solutions from various vendors created their own supportability issues, based on interoperability as well as multiple management tools and monitoring views.

2. Many customers expressed frustration over a support gap between protection/recovery solutions and the workloads themselves. We have often heard customers and partners complain that while their backup software reported successful backups and their recoveries reported complete, when they attempt to bring the data online, they either can’t or discover corruption.

· Calling the backup vendor, they are told that it must be the application.

· Calling Microsoft support, we provide best effort support but sometimes surmise that the data was not backed-up or restored properly.

There isn’t necessarily fault here, just the reality of multiple vendors with multiple approaches to solving the problem.

DPM 2007 was designed with these two scenarios in mind:

1. To provide a consistent and unified disk-to-disk-to-tape experience, that included near-continuous protection, routine tape backup and disaster recovery long-distance replication within a single protection product.

2. Provide the best possible backup and recovery solution that assured not only reliability, but also supportability.

Strategic Choices for how DPM does what it does

To do this, we did make some strategic choices. Because there were already several heterogeneous backup technologies that many customers believe fall short around complex Windows server application deployments – we chose to focus on the workloads that we are committed to, namely the Microsoft platforms and products: Windows Server, SQL Server, Exchange Server, SharePoint products and technologies, and our virtualization environments.

The other strong commitment we made was to use only the backup and recovery methods that are wholly supported by the product teams of the workloads we are protecting:

To protect data, we rely on the Volume Shadowcopy Services capabilities that Microsoft has been providing since Windows Server 2003 and the application servers of that timeframe. Almost every DPM protected workload uses a VSS writer that is provided by that product team, or other backup mechanism provided by the workload itself, as the best known and most supported way to secure that information -- as it was intended from those who know the inner-workings of the application itself. These are not hidden mechanisms or secret calls - they are well published as the intended mechanisms provided by the application as the intended way to back them up. But many legacy protection products choose to use other methods.

To restore data, we comply with the restoration capabilities and architecture of the application. In the case of Exchange, that means using the Recovery Storage Group – and specifically restoring the database into the RSG, and then using Exchange tools to selectively restore more granular objects. In the case of SharePoint, we use the ‘Recovery Farm’ capability that the SharePoint team developed for that purpose. In each case, our restore capability is tied to the functionality provided by that platform. Each protected workload has its own restore capabilities, and DPM is specifically designed to leverage them. Again, these mechanisms are well published, but not followed by some legacy protection products.

Supportability for Exchange protection solutions

While we are aware that there are some other backup products which offer a restore experience that is perceived as more integrated or transparent, this experience is made possible by reverse engineering the Exchange database and the use of other processes which are not supportable by the Exchange team. These approaches create a risk of database corruption in your Exchange installation and limits the Exchange team’s ability to fully support that installation should you encounter problems later.

Even before DPM was released, Microsoft saw significant issues with some backup technologies that tried to synthesize Exchange information stores to selectively restore data – and then push it directly back into the production database. This sometimes resulted in latent corruption that was only later discovered. The Exchange team published a number of KB articles detailing these risks and the associated support limitations they create, including:

KB 904845 addresses “Microsoft support policy for third-party products that modify or extract Exchange database contents”

Similarly, for disk-to-disk replication mechanisms, KB 895847 addresses “Multi-site data replication support for Exchange Server”

What is the future of Mailbox Recovery?

Looking towards the future, we are actively working with the application product teams to help them create innovative solutions that include restore capabilities that are either more direct (instead of the RSG/Recovery Farm), more granular (mailbox/message), or perhaps simply more transparent – but which are also fully supported – at which point, DPM will most happily deliver on that experience. We are making progress in that front. But innovation takes time.

In the near term, we have identified select partners that have been able to enhance the DPM backup capability with a more granular recovery mechanism, but in a way that is understood and approved of by the application product teams – e.g. Quest Recovery Manager for Exchange, enabling mailbox & message level restoration from the DPM restored replica. Details can be found on their site.

Finally, we also wanted to address some inaccuracies in the DPM Whitepaper on “How to Protect Exchange with DPM 2007”. The initial release inadvertently glossed over some of the details and did not reflect the full capabilities and processes for mailbox recovery correctly. When we learned that the paper was not providing clear guidance from some Exchange community folks and DPM customers, we immediately pulled the paper to correct it. We worked with the DPM engineering team as well as an Exchange MVP writer to provide better guidance, and it was re-posted last week at http://go.microsoft.com/fwlink/?LinkId=92497.

We hope this clarifies why DPM does what it does around Exchange protection to ensure that our customers have not only a great backup solution that protects Exchange in the manner that the Exchange developers believe that it should -- leveraging its VSS writer for fast block-based synchronization along with log backups, with additional benefits like offloading ESEUTIL and providing flexible protection options for CCR environments -- but also a strong and flexible recovery solution that allows for storage group, database and yes, mailbox, recovery in a way that is wholly supported and sustainable.

Jason Buffington
DPM Senior Technical Product Manager

Windows Storage Solutions

DPM 2007 -- Calculating Disk Allocation

dpm — Thu, 27 Mar 2008 15:36:00 GMT

When you create a new Protection Group using the “Create Protection Group” wizard, DPM internally computes and allocates volumes for storing the replica and the shadow copies. This blog post enumerates the various formulae that we use internally to generate the default size.

File system datasource

SC size = (dataSourceSize * recoveryRangeInDays * 2) / 100 + 1 GB;

Replica size = (dataSourceSize * 3) / 2;

SQL DB

Replica size = dataSourceSize * (1 + LogChange) / (AlertThreshold - .05);

SC size = 2.5 * recoveryRangeInDays * LogChange * dataSourceSize + 1 GB

Exchange SG

replicasize = dataSourceSize * (1 + LogChange) / (AlertThreshold - .05);

SCSize = 4.0 * recoveryRangeInDays * LogChange * dataSourceSize + 1GB

where

Log change = 0.06

Alertthreshold = 0.9

- Mukul Singh Shekawat

Protecting Exchange Server with DPM 2007 - new blog post from MS Exchange Team

JasonBuffington — Mon, 24 Mar 2008 22:37:00 GMT

The Microsoft Exchange team blog (msexchangeteam.com) recently posted an absolutely phenomenal blog post that describes exactly how DPM 2007 protects Exchange Server data. With soem really good explanations and nice graphics -- we might just reprint it as a new technical whitepaper (somewhat kidding).

Our hats are off to Doug Gowans, with help from Ruud Baars and Ben Appleby. Good stuff, guys !!

If you are looking to protect Exchange Server data -- take a look at their post.

http://msexchangeteam.com/archive/2008/03/19/448477.aspx

How does DPM really protect data?

JasonBuffington — Mon, 20 Aug 2007 23:35:00 GMT

This entry probably should have been a cross-posted ... but please check out my individual blog's latest post on how the DPM 2006 and DPM 2007 drivers work.

Also included is a 10 minute streaming video on how DPM 2007 uses VSS, as well as the native transaction logs of Exchange Server and SQL Server, as part of data protection.

Streaming Video - 9 minutes - How does DPM really protect data?