Let’s Build a Cloud… With PowerShell! – Part 2: Managing Virtual Machines

Let’s Build a Cloud… With PowerShell! – Part 2: Managing Virtual Machines

  • Comments 3
  • Likes

In the previous blog in this series, we discussed how to use PowerShell to deploy and configure Hyper-V and File Server clusters to set up a cloud infrastructure environment. We’ve seen how remote PowerShell invocation helps achieve that at scale and with zero manual intervention. We’ve also used a management server to host the PowerShell samples and invoke commands from it against the target servers being deployed, as a way to simulate how typical management tools are built.


In this blog post, we’ll continue on this path of automating common IT cloud management tasks, and take a look at a new set of sample scripts that demonstrate how to automate some of the key tasks around deploying, moving and monitoring virtual machines that are deployed onto the cloud infrastructure we’ve built.


The cloud infrastructure we’ll be using as an example is the one which you’re already well familiar with, which was described in previous blog posts and is configured by the samples in the previous sample package. It basically looks like this:




In that simplified environment, we basically have a library of VHD images that you can use to provision virtual machines from. The samples provide an example on how to provision virtual machines from the library, and configure them for high availability and in a way that when these virtual machines run, Hyper-V would store the running VMs on the same file server in the VMLibrary share, using the new support for Hyper-V over SMB3.0. By the way, Jose Barreto has a really cool blog showing multiple ways for rapid provisioning VMs from an SMB share!. You should feel free to mix and match these techniques with the ones discussed here and optimize the scripts to your needs.


Lets now dive into the specific samples in the new package and discuss each of them and some of the operations they demonstrate. For each sample, we’ll discuss what it does, how it does it, and take a look at the specific cmdlets that it uses to achieve the task at hand.


This is the basic virtual machine deployment script. At the heart of it is really a very simple flow: It takes a VHD from the library, creates a new image that will be used for the running VM, places the new virtual machine on the cluster, configures the necessary networking for it, and triggers monitoring on that virtual machine so that we can later (in MonitorVM.ps1) calculate chargeback information.


Let’s take a deeper look into the script. The script starts by getting a few arguments that will allow you to tweak what it’ll do. Naturally it gets the name of the VM to be provisioned, but it also gets a few input arguments to know which virtual machine to deploy and on what cluster, along with a few characteristics of the virtual machine (like the memory size it needs). You can also tell it how the library VHD should be instantiated (copied over or linked back to the library using a differentiating disk).


The first thing that the script does is to select which node to place the virtual machine on. Usually, in the process of a virtual machine placement onto a cloud, there needs to be logic as to which host is the best fit for that VM, or which host is least loaded. In the sample, we just chose to implement a random algorithm to spread the VMs across the cluster. In real life– you’d likely want a smarter algorithm that fits your specific need (or you could just use System Center Virtual Machine Manager and let it do the smart thing!). In the following snippet you can see how the script uses Get-Cluster | Get-ClusterNode to build a list of all nodes in the cluster, and then uses Get-Random to just select one of them as the target:



#Find a random node to place the vm on

$Nodes = @(Invoke-Command -Authentication Credssp -credential $Credential -ComputerName $ComputerName -ScriptBlock {

       param( $Name )

             Import-Module FailoverClusters

             Get-Cluster -Name $Name| Get-ClusterNode

          } -ArgumentList $Computername)                 

if ( -not $Nodes)


throw "No nodes found"



$Target = Get-Random -Maximum $Nodes.count -SetSeed (get-date).millisecond 


$TargetName = $nodes[ $target ].name 



Once we have the node selected and stored in $TargetName, we can now get to the task of provisioning the actual VM. The script always creates the VHD as a differentiating disk from the parent VHD which is in the master library of VHDs (to avoid duplicating the base), but you can specify that you want the parent VHD to be copied over so that the VM won’t depend on having the parent kept in the library. This all leads to a call to New-VHD to actually create the new VHD for the virtual machine.



# construct the path for the new differencing disk

$parent = Get-ChildItem $ImagePath


# Change vhd name if there is a name collision

$ChildPath = Join-path -Path $defaultVHDLocation -ChildPath ( $Name+ $parent.extension )

$ChildPath = UniqueVHDPath -Path $ChildPath


if( $copyParent ){

$NewParentPath = UniqueVHDPath -path ( Join-path -Path $DefaultVHDLocation-ChildPath $Parent.Name )

Copy-Item -path $parent -Destination $NewParentPath

$parent = Get-ChildItem $newParentpath

if(-not $?){throw "VHD could not be copied"}



$NewVHD = New-VHD -ParentPath $Parent.fullname -Path $Childpath -Differencing


Once this is done, it’s a simple matter of a New-VM call to create the new virtual machine and connect it to the network through a virtual switch. If you recall, in the deployment and configuration scripts, when a Hyper-V node was created, we also instantiated a virtual switch that will be used to connect VMs to the network. In a real environment, you may have cases where you need to instantiate more than one switch on a given host, but for simplicity we assume a single switch per host on this configuration:


$vm = New-VM -name $Name -VHDPath $NewVHD.path -MemoryStartupBytes(1 * $Memory) –switchname $SwitchName




Now comes a tricky part. If you look at the sample, you’ll see that right after creating the VM, the sample can use the input argument called NICSettings to set up some advanced settings on the VMNic that was just created (think of the VMNic as the port in the virtual switch that the VM is connected to). This input argument is a PowerShell hashtable and it’s intended to be used only when it’s being called by another script. We will actually use it when we get to our next blog and set of samples of configuring virtual machines with network virtualization, so for now, we’ll leave this piece of code as a mystery code…


That’s about it. Once the VM is there, the script triggers monitoring of the virtual machine (by calling MonitorVM.ps1, see below), and then clusters it by calling Add-ClusterVirtualMachineRole and starts it using Start-VM.


Deploying virtual machines with Windows Server 2012 is pretty easy, isn’t it?


Migrating virtual machines around the datacenter (or within a cluster) in a planned manner is a common task for the cloud admin. You’d usually do it for the purpose of balancing the load on your servers, or for servicing servers without interruption to the running workloads. Note that specifically for the servicing scenario, Windows Server 2012 actually has a new capability called “Cluster Aware Updating” that will do all of this heavy lifting for you!


Anyway, for the purpose of the migration sample, we just move a VM between two nodes as specified, so this sample is pretty basic and demonstrates the use of the Move-ClusterVirtualMachineRole cmdlet. Most other samples you have probably seen where focused on Move-VM, which migrates a VM in a non-clustered environment, so we thought it’s worthwhile to provide it just to make the point that it’s different when it’s done on a cluster.


This is the last but certainly not the least among this set of samples. This sample demonstrates the use of yet another new Windows Server 2012 feature called resource meters. What resource meter do is allow you to place ‘meters’ on a provisioned virtual machine, start metering it’s network, compute and storage activities, and collect information about resources being used so that later you can use it to calculate how much this virtual should cost to the owner of it.


These resource meters are kept together with the virtual machine even as it’s being migrated across the datacenter, so if you’re building chargeback tools, all you have to do is enable metering on a provisioned virtual machine, and on a periodical basis go and read those meters, calculate the cost information based on that, and reset them for the next cycle. Pretty cool, ha?!


This sample has two modes of operation. It can initialize metering, which is actually happening when the OnboardVM.ps1 scripts calls it while provisioning a VM, and then it has a reporting mode, where you can run it on a VM that was previously provisioned and it will produce a simplified cost report for that VM.  Here’s the script’s main dispatching code. Note the use of Invoke-Command –ScriptBlock to actually pass a function of the script as a script block to the target host, instead of having the two modes implemented in separate script files:



if( $Initialize )


   # Set up monitoring

   if ($computername)


        Invoke-Command –ComputerName $computername -ScriptBlock( get-item function:\Start-VMMonitoring).ScriptBlock –ArgumentList $VM,$ACLs




        Start-VMMonitoring $VM







        invoke-commandComputerName $computername -ScriptBlock( get-item function:\Calculate-VMCost).scriptblock -ArgumentList$VM, $VMSetting




        # Calculate the VMCost for the chargeback cycle

        $data =Calculate-VMCost  $VM 

        Write-host "Total Cost: $($data.TotalCost)"





To initializing metering, there are actually two things that need to happen. The first is to define meters on network traffic in and out of the virtual machine. Those meters are actually implemented as ACLs on the VM Network Adapter and they can be filtered to only measure “interesting” network traffic flows. Enabling network metering is done by calling the Add-VMNetworkAdapterAclon the VM Network Adapter, and with an action of “Meter” instead of the usual allow/deny you would expect from an ACL.


The sample also takes a few parameters from a settings file to filter the metering and apply it only traffic the virtual machine generates on the tenants network subnet, filtering out traffic to a potential storage volume, for example. Other examples for such filtering would be to only measure network traffic to/from the Internet and not measure internal traffic between virtual machines.


The second thing that needs to be done is to  call Enable-VMResourceMetering on the VM. That will trigger metering of the CPU usage and disk usage of that VM. Here’s the code snippet from the sample that actually initializes metering on the virtual machine:



# VM netadapter ACLs for metering

$ACLs  = $VMSetting.ACLs


Function Start-VMMonitoring








   $NewVM =Get-VM $VM


   #Set ACLs for metering inbound network traffic

   $NewVM |Get-VMNetworkAdapter |Add-VMNetworkAdapterAcl -Action Meter -Direction Inbound -RemoteIPAddress $ACLs.Meter.inbound

   if(-not $?){ throw"ACLs could not be set on $($NewVM.name)"}


   #Set ACLs for metering outbound network traffic

   $NewVM |Get-VMNetworkAdapter |Add-VMNetworkAdapterAcl -Action Meter -Direction Outbound -RemoteIPAddress  $ACLs.Meter.outbound

   if(-not$?){ throw"ACLs could not be set on $(NewVM.name)"}


   #Enable resource metering

   $NewVM |Enable-VMResourceMetering

   if(-not $?){ throw"Resources are not being metered on $(NewVM.name)"} 





Now, when it comes to collecting the measured data, all the needs to happen is a call to Measure-VM and then reading the returned data:



function Calculate-VMCost









        $Acls =$VMSetting.ACLs

        $Rates =$VMSetting.Rates



   # Get the VMMeteringReport

   $ChargeBackData = Get-VM $VM | Measure-VM


   . . .




The rest is code that looks at $ChargeBackData and does some dummy calculation of cost for each resource based on rate information found in the settings file. Once we’re done reading, we want to reset the meters so that we’re ready for the next charge cycle:


# Reset metering for next bill cycle. (No output)

Get-VM $VM | Reset-VMResourceMetering


And that’s all there is to it!


The Settings File

You knew this is coming, right? Smile


As in the deployment and configuration sample set, the samples are built in a generic way that should allow you to easily reuse them and tweak the right settings to make them fit your needs. In this case, the configurable settings include things like the VHDLibrary location, IP addresses of your cluster, and some additional settings. Here’s a snapshot of some of them:



$FileServerName = 'Scaleoutfs'

$VHDlibraryShareName = 'VHDLibrary'


$VHDLibraryLocation = "\\$FileServerName\$VHDlibraryShareName"



$VSwitchname = "TenantSwitch"

$AdminAccount = "hcp\Administrator"




That’s about it for now. In the next blog, we’ll take a look at a set of advanced virtual machines on-boarding samples that will show how to deploy virtual machines in an environment with Hyper-V network virtualization enabled. Stay tuned!

And last but not least, I’d like to remind you that if you find the scripts useful, you should feel free to contribute your own modified versions back to the TechNet ScriptCenter. It’s all about the community!


One PowerShell to Rule Them All… Smile


Yigal Edery



Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment