John Howard - Senior Program Manager in the Hyper-V team at Microsoft

Senior Program Manager, Hyper-V team, Windows Core Operating System Division.

Automating VM Compacting and Archiving

Automating VM Compacting and Archiving

  • Comments 2
  • Likes

I apologise in advance - this is probably the *WORST* bit of scripting I have ever written. However, this is stage one and no doubt it will undergo many months of optimisation before I'm finished - it does the job, that's about it.

As I alluded to a few weeks ago, I tend to go through a monthly routine of defragging/pre-compacting/compacting/zipping my VMs. Although partly automated, I've been playing with a mechanism to get the VM and the host to play nicely together and go through the routine in a fully automated way. This is in addition, of course, to nightly backups of user data, exchange data etc - this is just to make sure I can come in on a Sat morning and backup a load of compact ZIPs to a DVD for disaster recovery purposes.

The biggest problem in solving the fully automated problem is that of synchronising and signalling between the guest and host. However, Windows Server 2003 gives you that for free if you know where to look, without needing to write any custom code (apart from the action itself).

The end-to-end plan looks like this:

In the guest

- Create a weekly/monthly scheduled task
- Stops all services to free up in-use files but keep the machine running
- Defrags each hard disk
- Runs the VS Precompactor in "Silent" mode
- Starts the services which were stopped (optional due to next step)
- Signals the host machine to say I'm ready to do the compacting
- Shutdown, forcing open applications to close

On the host

- Wait for the signal
- Wait for the VM to shutdown
- Compact each hard disk of interest [See note]
- Add each compacted hard disk to a new archive
- Start the VM.

Note: I tend to use a seperate swap VHD, so there's no point compacting or archiving these

The signalling comes in the form of EventCreate and EventTriggers. EventCreate is a bit of VBScript which writes and event to the event log, both locally or remotely, and using alternate credentials if necessary - the remote bit being ideal here. EventTriggers allows a machine to watch the eventlog and call a program/script once that event is seen.

So lets put this into practice. On each guest (a Virtual Domain Controller, or "VDC" in this case), I have a c:\defrag directory containing Dave's defrag tools I mentioned a little while back, and a defragandcompacttask.cmd script containing the following:

<stop services>
c:\defrag\defrag -d c:
c:\defrag\defrag -d n:
c:\precompact\precompact -Silent
eventcreate /T WARNING /D "VDC Ready for compaction" /ID 777 /SO John /L Application /S [HOST] /U [domain\user] /P [password]
shutdown /s /f /m \\vdc /t 0 /c "Precompact"

The first line is custom to the VM and consists of a series of "net stop <service>"
The next two lines are hopefully obvious: Defragging the C (system) and N (NTDS) drives. Customise as appropriate.
The next line is similarly obvious - it calls the Virtual Server precompactor in silent mode.
The next line creates an event on the VS host. It's the "777", an arbitrary figure I chose for this VM, which is the trigger. The other parameters to eventcreate are for a WARNING level entry, a description, the Source (makes filtering easier), the application log and the host and credentials. Note that as this is in free-text, I created a basic user account with limited privilege in AD to create the log entry on the host.
Lastly, the script forces a shutdown.

At this point, there will be a delay while the VM shutsdown.

Lets move onto the host. Again, apologies for the worst form of scripting I've ever hacked together. The first thing you need is the trigger picking up event 777 in this case. As I have multiple VMs, I have a createtriggers.cmd script which re-creates the triggers if necessary, containing lots of similar lines - each one varies the event ID and the VM Name (again, this was VDC, so it should be obvious).

eventtriggers /create /L Application /T Warning /EID 777 /TK b:\backups\vms\777.vdc.vbs /tr vdccompact /so John

This script will fire when it sees an event like below:


The next thing is the 777.vdc.vbs VBScript where all the heavy work happens. I'll annotate it throughout hopefully without having broken it along the way :)

[Change the name of the VM Here]
Const VM="VDC"
Const BackupDir="b:\backups\vms"

[Change to an appropriate archiver here]
Const YourArchiver="<path_to_your_archiver eg winrar or winzip.exe>"

[These come straight out of the VS programmers help]
Const vmVMState_TurnedOff = 1
Const vmVMState_Running = 5

szDate = Year(now)&"-"&month(now)&"-"&day(now)
szArchiveName = BackupDir&"\"&szDate&" "&VM&".rar"
set objShell = CreateObject("")

[This is a common routine so that I can log progress through the hosts event log]
Sub LogEvent(lID, szData)
    szCmd = chr(34) & "EventCreate" & chr(34) & " /T INFORMATION /D " & _
            chr(34) & szData & chr(34) & " /ID " & lID & _
            " /SO John /L Application"
    objShell.Exec szCmd
End Sub

LogEvent 600, VM & " precompact trigger received"

[Now we start: We need a reference to the VM Object
Set objVS = CreateObject("VirtualServer.Application")
Set objVM = objVS.FindVirtualMachine(VM)

[Once it's shutdown, call compact and add to archive]
if 0 = WaitForShutdown() then
    Call Compact
    Call AddToArchive
end if

[Start the VM Running]
LogEvent 612, "Starting " & VM

[Wait for it to start, logging an error if it fails]
i = 0
while (i < 100) & objVM.State <> vmVMState_Running
    LogEvent 613, "Waiting for " & VM & " to enter running state"
    wscript.sleep 10000
if objVMState <> vmVMStateRunning then
    szCmd = chr(34) & "EventCreate" & chr(34) & " /T ERROR /D " & _
            chr(34) & "VM " & VM & " did not restart." & chr(34) & _
            " /ID 604 /SO John /L Application"
    objShell.Exec szCmd
    LogEvent 615, "Compaction of " & VM & " completed :)"
end if  

[Hopefully fairly straight forward]
Function WaitForShutdown
    WaitForShutdown = 0
    iCount = 0
    while (objVM.State <> vmVMState_TurnedOff) and (iCount < 100)
       LogEvent 601, "Waiting for VM " & VM & " to shutdown"
       wscript.sleep 10000
       i = i + 1

    ' Error if VM did not shut-down
    if (objVM.State <> vmVMState_TurnedOff) then
        WaitForShutdown = -1
        szCmd = chr(34) & "EventCreate" & chr(34) & " /T ERROR & _
                /D " & chr(34) & "VM " & VM & _
                " did not shut down! Compact failed." & chr(34) & _
                " /ID 601 /SO John /L Application"
        objShell.Exec szCmd
    end if   
end Function

[Where the compaction actually happens]
Sub Compact()
      LogEvent 603, "Compaction for VM " & VM & " starting"


[Get a reference to each Hard Disk connected]
      set colHDConxns = objVM.HardDiskConnections


[Loop through each VHD]
      for each objHDConxn in colHDConxns
[Get a reference to the hard disk itself]
          set objHD = objHDConxn.HardDisk


[I'm not interested in the swap file, or the big WSUS data VHD]
          if 0=instr(1,lcase(objHD.File),"swap") and _
             0=instr(1,lcase(objHD.File),"wsus data") then
              LogEvent 605, "Compacting " & objHD.File & ". " & _
                            "SizeInGuest: " & objHD.SizeInGuest & " " & _
                            "SizeOnHost: " & objHD.SizeOnHost

              iBefore = objHD.SizeOnHost


[Start a compaction task running, and report progress periodically]
              set objTask = objHD.Compact
              while not(objTask.IsComplete)
                  LogEvent 606, "Compacting " & objHD.File & " " & _
                           objTask.PercentCompleted & "%"
                  wscript.sleep 10000
              LogEvent 607, objHD.File & " compacted. SizeOnHost: " & _
              LogEvent 604, "Ignoring compaction on " & objHD.File
          end if
end sub

[Hopefully self evident apart from why I have to set objVM=Nothing near the top.
VS locks the VMS otherwise]
Sub AddToArchive()

    ' Add the VMC to the archive
    szCmd = chr(34) & YourArchiver & chr(34) & " A " & chr(34) & szDate & _
            " " & VM & chr(34) & " " & chr(34) & objVM.File & chr(34)
    LogEvent 610, "Adding " & objVM.File & " to " & szDate & VM
    set objVM=Nothing ' Need to do this as VMC is locked othersise
    set wsx=objShell.Exec(szCmd)
    while wsx.Status = 0
        wscript.sleep 1000
    set objVM=objVS.FindVirtualMachine(VM)

      ' Add each VHD we're interested in to the archive
      set colHDConxns = objVM.HardDiskConnections
      for each objHDConxn in colHDConxns
          set objHD = objHDConxn.HardDisk
          if 0=instr(1,lcase(objHD.File),"swap") and _
             0=instr(1,lcase(objHD.File),"wsus data") then
              LogEvent 610, "Adding " & objHD.File & " to " & szDate & " " & VM
              szCmd = chr(34) & YourArchiver & chr(34) & " A " & szDate & _
                      " " & VM & chr(34) & objHD.File & chr(34)
              wscript.echo "Executing " & szCmd
              set wsx=objShell.Exec(szCmd)
              while wsx.Status = 0
                   wscript.sleep 10000
          end if
end sub

And that's it
Hope this is useful


  • That is very impressive, but wouldn't it be easier to use vmount on the host to defrag and precompact the disks?

    I know that means that the host has to be able to mount guest VMs, but there would then be nothing to do on the guest but shut it down, so eveything could be scheduled on the host with no need for signalling, and no need to customise the guest at all.

    Or is that back to front & you would rather everything could be done on the guest?


  • Dave - I thought about using the VHD Mount capability but the main reason for not using it is that it's still beta... not that I've heard of or had any problems. But, when playing with VHDs in what is, although a home toy environment, effectively production, I'm still cautious. So I guess when SP1 RTMs next year, I'll revisit this for sure.  Great feedback though - much appreciated - thanks.


Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment