virtualboy blog

Matt McSpirit on Virtualisation, Management and Core Infrastructure

Cool Tools: VHDCopy, VHDCopEE and VMProv

Cool Tools: VHDCopy, VHDCopEE and VMProv

  • Comments 14
  • Likes

If I said there was a tool that could accelerate fixed-sized VHD copies by 10 times, would you be interested?  If I said you could instantly provision and boot a new VM from a currently copying VHD file, would you be interested?  Well, if the answer is yes, to either of those questions, then I have a couple of tools that you may find useful.

Hat-tip to Dilip, one of our MVPs for File Systems/Storage and author of ‘Inside Windows Storage, for providing the link to VMUtil.

VHDCopy

When you think about it, a fixed-sized VHD, typically, contains a lot of nothing.  What I mean by that is, think about it, if you have a 100GB fixed-size VHD, and inside that VHD, you’re only using say, 30GB, why should you have to transfer (for whatever reason) the full 100GB, when only 30GB is actually useful stuff?  You could apply the same example to a backup drive – if I had a new backup drive of 1TB, and I’d only backed up 50GB so far, and I needed to move the files somewhere else, what would be quicker to transfer, 50GB or 1TB?

This is just one of many inefficiencies that exist within a file copy of a VM, that VHDCopy solves, by refraining from copying the meaningless portions that exist within a VHD.

Advantages include:

  • Significant reduction in disk I/O - typical savings may range from 20% to 90% reduction in disk I/O - your mileage will vary
  • Significant reduction in file copy time - typically order of 2X to 10X faster - your mileage will vary
  • Scriptable command line tool that requires no installation
  • Use VHDCopy to migrate from Windows Server 2008 based Hyper-V to Windows Server 2008 R2 based Hyper-V and its Cluster Shared Volumes (CSV)

There’s a datasheet here for more information, and you can always head over to the VHDCopy page for more stuff, including download links.

VHDCopEE

Building on VHDCopy, but providing more functionality, particularly around network transfers, as you can see from the table below:

image

Although it’s not quite clear from the website, depending on the page you’re looking at, both VHDCopy and VHDCopEE do run on the latest Windows OS’s, both Server and Desktop, yet VHDCopy will only accelerate local transfers, rather than network transfers, for which you’ll need VHDCopEE.

Advantages include:

  • Significant reduction in disk I/O of up to 90% depending upon VHD contents
  • Significant reduction in file copy time - up to 10 X faster based on VHD file contents
  • Scriptable command line tool that requires no installation
  • Use VHDCopy to migrate from Windows Server 2008 based Hyper-V to Windows Server 2008 R2 based Hyper-V and its Cluster Shared Volumes (CSV)
  • Use VHDCopEE with Microsoft System Center Virtual Machine Manager 2008 R2 and its Rapid Provisioning feature
  • Get SAN storage features such as Instant Clones and Instant Provisioning while using low cost non SAN storage

More on the ‘bolded’ ones later…

Again, there’s a datasheet here, and you can head over to the VHDCopEE page for more info.  You can grab the download here.

VMProv

This one’s pretty darn clever too.  Think about it, you want to create a VM from say, a VHD that already exists.  So, you copy, and past the VHD, and it takes x-minutes to copy.  Using the above technologies, it’ll be quicker, but we’re still going to have to wait until the copy has finished to start the VM, right?  Wrong.

VMProv allows a VM to be provisioned and available for use within seconds, well before the accelerated file copy has finished. Once the file copy finishes, the provisioned VM is no different than if it had been utilized after the file copy operation had finished. While the file copy is happening, the VM is fully available, and you may install any updates or new software. All changes made to the VM while the file copy is happening are preserved.

Cool hey?

On top of that, and going back to the bolded points under VHDCopEE, VMprov is designed to be used with Microsoft System Center Virtual Machine Manager R2 and its Rapid Provisioning feature.

scvmm_soln

Think about using SCVMM – whenever you’re sending out a VHD from the library, it’s typically going to be a file copy process, and the bigger the file, the longer it will take.  Using VHDCopEE will speed up the transfer, and combining that with VMProv will mean you can start the VM up without waiting for the VHD to finish copying.  Combine that with a Rapid Provisioning PowerShell script, found here, and you’re going to accelerate stuff considerably.

You can read the VMProv datasheet here, and check out the VMProv page for further info.  If you want to download it, you can, here.

That’s all from me – make sure you check out the tools if you get chance!



Comments
  • And what about the cost of the tools? I couldn´ find anything in VMUtil.com

  • I take it that these tools do not zero out the empty space of the VHD and that's why they are quicker.  That would be fine in a lab but possibly not for a production environment.

  • Hi KayZer - not sure on pricing, I'm just checking with Dilip, but you may want to ping him a message to check - mailto:info@vmutil.com.

    Also, the versions currently posted are capped at 128GB, but uncapped versions are available - you just need to contact Dilip, again using mailto:info@vmutil.com.

    @Aiden - On the contrary - if you use VHDCopy with the [/Secure] flag, it will ensure the

    destination VHD file will have zeroed data blocks for the blocks not being currently used by

    NTFS. The option is ignored if the file system inside the VHD file is not NTFS.  Not sure how this affects the speed of copy, or if it's something that's carried out after the transfer.  The user guide (http://www.vmutil.com/documents/VHDCopy_UserGuide.pdf) may have more info, but I couldn't see anything standing out.

    Hope that helps!

    Matt

  • @Aidan - apologies for my @Aiden typo.

    Matt

  • ok when is MS going to buy the rights to the product?  These are the things will help sell hyper-v r2.

  • Hi TonyR - great point, and I'm sure you're not alone in your thoughts.  Customer demand for features like this drive what goes into the next releases of the technology, so watch this space I guess!

    Matt

  • Another company solving the VHD issue as well as other hyper-v / storage related issues is www.virsto.com

  • Great comment Steve - much appreciated, and keep them coming!

  • http://www.virsto.com/products/

    "We want to get farther into customer beta test before talking specifically about our technology. We've been working hard on it for over two years, so believe us, we can't wait to talk about it. But there's no point in yakking about the product until it's been verified it in the real world.

    We can say that we've designed our technology with the following ideas in mind."

    Will bo good to see the release products..

  • True - maybe some more people testing will help them along with the process! ;-)

  • ok they need to not have these things not expire april 1st, kinda gives me the willies just thinking about the possiblities

    tr

  • VMUtil hopes to lift the Beta tag on these products in early March 2010. Until then at least, the products are free.

    send comments to info@vmutil.com

    Dilip Naik

  • The product has more to it than just skipping the empty parts of the VHD file. It will certainly avoid reading the zeros, and will optionally write the zeros. But in addition, optionally even allocated, but "useless" parts of the VM will be skipped.

    A typical scenario might be create an OS master, copy it, install SQL and declare the result to be a SQL master image. In this scenario, you would write a bunch of zeros, then overwrite them with SQL stuff. Because the VM can be booted quickly, SQL writes will happen first, and then this occupied data need not be zero filled

    And yes, the more people that use it, the quicker the issues will shake out. Very likely that the Beta tag will be lifted in early March 2010

    Dilip

  • Hi guys,

    vmutil.com is no more available. Can anyone provide a download link for the tools above ?

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment