Learn about Windows PowerShell
Summary: Guest blogger, Keith Mayer, talks about using Windows PowerShell to manage cloud-based backups.
Microsoft Scripting Guy, Ed Wilson, is here. Welcome back Keith Mayer as our guest blogger today. This is Part 1 of a two-part series. In Part 2, we’ll restore virtual machines in Windows Azure from these backups.
Keith Mayer is a senior technical evangelist at Microsoft, and he focuses on the Windows infrastructure, datacenter virtualization, systems management, and the private cloud. Keith has over 20 years of experience as a technical leader of complex IT projects, in diverse roles, including network engineer, IT manager, technical instructor, and consultant. He has consulted and trained thousands of IT professionals worldwide on the design and implementation of enterprise technology solutions.
You can find Keith online at http://KeithMayer.com.
Windows Azure Infrastructure Services provides the ability to easily provision or migrate storage, virtual machines, and virtual networks to the global Windows Azure cloud platform by using a cost-effective Pay-As-You-Go model. In my prior Weekend Scripter posts, Getting Started with Windows Azure and PowerShell and Remoting the Cloud with Windows Azure and PowerShell, I provided an introduction to using Windows PowerShell for automated provisioning of Windows Azure cloud fabric resources and workloads running inside virtual machines in Windows Azure.
Once you’ve provisioned a few lab virtual machines on Windows Azure, you’ll likely want to create a backup copy of each virtual machine. This is particularly useful in test lab scenarios, so that you can quickly revert virtual machines back to a known state prior to performing a set of tests.
When you are working in the on-premises world, this is typically performed via virtual machine backup, snapshot, or checkpoint capabilities. In Windows Azure, back up and restore of virtual hard disks can be quickly performed in the cloud with the Windows Azure PowerShell Module by leveraging the Start-AzureStorageBlobCopy cmdlet. However, there’s a bit of work we’ll need to do upfront to capture the information we’ll need for completing this process.
In this post, we’ll step through the process of building a Windows PowerShell script to back up virtual machines in Windows Azure. At the end of this post, I’ve provided a link to the next step, restoring virtual machines in Windows Azure, so that you’ll have the complete end-to-end process.
To back up virtual machines in Windows Azure, we’ll step through the following tasks:
Note To learn more about the basics of Windows Azure Infrastructure Services, you might also be interested in the “Early Experts” Cloud Quest and our scenario-based Cloud Labs step-by-step guides. Both are free online study resources that provide hands-on lab exercises for leveraging Windows Azure and building key IT pro cloud scenarios.
The virtual machine you want to back up and restore can be selected by using the Get-AzureVM cmdlet. Running Get-AzureVM alone returns a list of virtual machines that are currently provisioned in Windows Azure.
Get-AzureVM cmdlet output
To select a particular virtual machine, you can pass the ServiceName and Name values as parameters and set the output to a new Windows PowerShell variable.
Selecting a Windows Azure virtual machine
Now our selected Windows Azure virtual machine can be referenced by using the variable $vm in the remainder of our script.
To capture a valid backup of each virtual hard disk, we also need to temporarily shut down the virtual machine to a state where the virtual machine is not running, but its configuration is kept in a provisioned state. We can accomplish this with the Stop-AzureVM cmdlet.
Using Stop-AzureVM with the StayProvisioned parameter
Now that our virtual machine is selected and in the right state, we can proceed to the next step of finding each virtual hard disk we want to back up and restore.
Virtual machines in Windows Azure can be provisioned with two general types of virtual hard disks: operating system disks and data disks. Each virtual machine will have one operating system disk from which it boots and runs the operating system. In addition, each virtual machine can have one or more additional data disks on which program code and data files can be stored. To perform a complete virtual machine backup, we’ll need to locate all of the virtual hard disks that our virtual machine is currently using.
To store the location for the operating system disk, we can use the Get-AzureOSDisk cmdlet.
Locating the virtual machine operating system disk with Get-AzureOSDisk
For any virtual hard disk that we want to back up or restore, the two property values in which we’ll be most interested are the DiskName and MediaLink values, which are shown in the following image. These values provide the information that we’ll need to properly back up and restore each virtual hard disk that is associated with a virtual machine.
Common property values for a Windows Azure virtual hard disk
To store the location for all data disks, we can use the Get-AzureDataDisk cmdlet. Because virtual machines can be provisioned with multiple data disks, this cmdlet returns of a collection of data virtual hard disks.
Storing location for data disks with the Get-AzureDataDisk cmdlet
Prior to performing a backup, we’ll need to make sure that a container exists in our Windows Azure Storage Account to store these backup copies. First, we’ll need to determine the name of our Windows Azure Storage Account. We can do this by leveraging the MediaLink property of Azure Disks mentioned earlier.
Determining the name of Windows Azure Storage Account by using MediaLink property
Now that we know the name of our Windows Azure Storage Account, we’ll want to set it as the current storage account for the remainder of our script by using the Set-AzureSubscription cmdlet.
Setting the current storage account
Next, we can easily check to see if our desired container location for storing backups already exists inside our storage account, and if not, we can quickly create it by using the New-AzureStorageContainer cmdlet.
Creating a new Windows Azure storage container
We can confirm that the new storage container has been created by using the Get-AzureStorageContainer cmdlet without parameters.
Confirming creation of storage container with Get-AzureStorageContainer cmdlet
Now, we’re ready to back up our virtual machine!
To create a backup copy of the operating system disk on our virtual machine, we’ll first set the values for a couple variables that identify the blob and container names for the virtual disk that we want to back up. Then, we’ll use the Start-AzureStorageBlobCopy cmdlet to begin the copy process to our previously defined backup container location.
Using Start-AzureStorageBlobCopy cmdlet to back up virtual hard disk
It’s important to note that the copy process performed by the Start-AzureStorageBlobCopy cmdlet is asynchronous in nature, and it runs in the background on the Windows Azure platform. To ensure that the copy process has completed before continuing with the next line in a script, we can use the Get-AzureStorageBlobCopyState cmdlet to wait until the copy process is finished.
Using Get-AzureStorageBlobCopyState to confirm that the copy process completed
To back up our Windows Azure data disks, we’ll use a similar set of cmdlets, but we’ll run them inside a ForEach loop because Windows Azure data disks are returned as a collection.
Back up data disks by using ForEach loop
After the backup process has completed, use the Get-AzureStorageBlob cmdlet to confirm that a copy of each virtual hard disk now exists in the backup storage container location.
Using Get-AzureStorageBlob to confirm backup copies
Our backup process is complete, and we can now restart the Windows Azure virtual machine by using the Start-AzureVM cmdlet.
Using Start-AzureVM to restart virtual machine after back up is complete
Congratulations! But keep learning!
You’ve completed the process for creating cloud backups of virtual machines in Windows Azure with Windows PowerShell! You can use the cmdlets and snippets in this post to quite easily build an automated approach to capture a backup of each Windows Azure virtual machine in your subscription, perhaps on a nightly basis.
In Part 2 of this series, we’ll walk through the process of restoring virtual machines in Windows Azure from these backups so that you can automate the complete end-to-end backup and restore process.
In addition, you may want to leverage these resources to continue your learning about Windows Azure Infrastructure Services:
Thank you, Keith, for sharing your time and knowledge.
I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at email@example.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.
Ed Wilson, Microsoft Scripting Guy
Hi Keith,Thank you for this precious memo.I am actually following your step by step. I also import my subscription to configure it by default but I couldn't be able to create a New-AzureContainer.I got instead the following error:Get-AzureStorageContainer : Can not find your azure storage credential. Please set current storage account using"Set-AzureSubscription" or set the "AZURE_STORAGE_CONNECTION_STRING" environment variable.At line:1 char:7+ if (!(Get-AzureStorageContainer -Name $backupContainerName -ErrorAction Silently ...+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidArgument: (:) [Get-AzureStorageContainer], ArgumentException + FullyQualifiedErrorId : ArgumentException,Microsoft.WindowsAzure.Commands.Storage.Blob.Cmdlet.GetAzureStorageCon tainerCommandIf I try to get-Subscription I can actually see that I have my 2 subscriptions.Does anyone has an idea for helping me to fix my issue ?Regards,Florian
I'd like to see one improvement of the scripts. Because currently $vmOSBlobName is derived from $vmOSDisk.MediaLink.Segments[-1], next time when the backup scripts are run, we are prompted to overwrite the .vhd file in the backups container. So we can only
keep one backup copy...
I desire to backup multiple copies of the same .vhd file, of different timestamps, and be able to select one of the multiple copies to restore. Basically I want to be able to do the same thing as I have been doing with the GUI in Hyper-V.
The 'Get-AzureStorageContainer' didn't work for me either with the same error as Florian has reported. What works instead is these 2 commands:
$context = New-AzureStorageContext -StorageAccountName $storageAccountName -StorageAccountKey
New-AzureStorageContainer -name $backupContainerName -Permission Off -Context $context
The comes from STORAGE >> >> MANAGE ACCESS KEYS in the Azure management UI. Didn't try reading the key via script though - if this cannot be done I'd just stick it into a constant in the script. HTH
I want to be able to not shut down the VM as we are live and this would be utilized as a backup solution. Is there a way to not shutdown the VM?
Hello, Ed, and thank you for that article I used to schedule my backup on automation. But I met an issue I'm trying to figure with MS support team : the Start-AzureStorageBlobCopy cmdlet will not erase an existing file with the same name, even by using
he "-force" switch. So, if you want to run that operation everiday days/weeks/whatever, you will meet an issue, and the command won't eventually return any error (copystatus=success). I wanted to rename the previous copy, but PowerShell doesn't provide any
command to do so, and the option to delete previous copy before making a new one is a bit dangerous on a backup scenario.
If anyone has an idea, I'd appreciate; if MS Support Team bring me one, I'll try to think about sharing it here.
Question re permissions. I am trying to work out how to do this with a user that only has the ability to write (not read, delete or modify) blob data (like IAM in AWS). This would allow me to place the scripts on the machines themselves without fear of
them overwriting historical snapshots if they became compromised. Any pointers ?
Scripting to get a snapshot, that's just silly. In VMware it's right-mouse-click > 'take snapshot'
Good Article Thanks for sharing.
Did the Part 2 (how to restore) of this series ever come out? I'm looking into backup methods. Snapshots seemed easy, but I didn't like the restore process, which involves overwriting the original disks. Making a copy of the disks via BlobCopy seems like
a good idea, but I think I can skip a lot of the scripting with a single save-azurevmimage command, and it works even if the vm is still online.
Of course, the hard part now is the restore. So far, my process is to create a new vm from an image backup, delete the new vm but keep the attached disk, and then attach the disk to another vm to recover files or to swap the place of the os or data disk.
I have a question on this - it is very helpful but I am not great with powershell and what I am looking to do is quite complex in my opinion.
The Aim : automate a task to control the size of a VM in azure - the code is below, it is fully tested and also fully functional and it works fantastically. However, what I would like to do - before I do the step "Update-AzureVM" - is take a snapshot blob back-up
of the VM.
My plan is as follows :
Change instance Size using below code (done)
Take backup to blob storage in Azure
Can anyone help elaborate my code?? Please
P.s. this is only one of the sizes for demo purposes.
Select-AzureSubscription -SubscriptionName "Microsoft Azure Enterprise"
[string]$ServiceName = "",
[string]$VMSize = "Large"
Get-AzureVM –ServiceName $ServiceName –Name $Name |
Set-AzureVMSize $VMSize |
HowTo-SetAzureVMSize –ServiceName "" –VMSize "Large"
The above code lost it's formatting upon posting, but you get the idea
@Florian - I realise this is an old post, but it might help others with the same issue. You need to use Set-AzureSubscription to set your CurrentStorageAccountName for the current session, which you do like so:
Set-AzureSubscription -SubscriptionName "My Subscription" -CurrentStorageAccountName "MyStorageAccountName"
You should also, as a matter of good practice, use:
Select-AzureSubscription -SubscriptionName "My Subscription"
before doing that, so that you have a currently selected subscription too.
And idea on the backup of an Azure VM when shutting down automatically with script??
Great post. Thank you.