Kevin Remde's IT Pro Weblog

How fast is fast? Virtual Machine Live Migration Improvements

How fast is fast? Virtual Machine Live Migration Improvements

  • Comments 4
  • Likes

When you’re doing a Live Migration** of a virtual machine between hyper-v hosts, you want it to go quickly.  You may be doing the migration of one or several or dozens of virtual machines all at once, and the performance of the network and the network paths you choose are going to determine how quickly you can get the job done.  Yes, sure, in one sense it doesn’t matter how long it takes if the VMs will continue to run and provide service during the migration.  But if I’m doing, say, an automated update of all of the hosts in my cluster, and allowing it to drive the live migrations of machines among hosts, the speed with which those migrations complete will ultimately determine how long it takes to complete the updates of all of those hosts.  If I’m really maxing out the capabilities of Hyper-V in Server 2012 R2 or Hyper-V Server 2012 R2, that could mean as many as 8,000 virtual machines moving around and among 64 clustered hypervisor nodes.  So, speed is still important.

In the past, memory of a running virtual machine was just sent over the wire (TCP/IP) as it was.  Nothing special was done to it.  But as hardware costs have improved to support larger and larger scale, and as we’re afforded the ability to run more virtual machines with more and more memory, we certainly want to do everything we can to make that transfer of memory and configuration data go as quickly as possible.  So to address this and improve things, we’ve added two new technologies to hyper-v in Windows Server 2012 R2 and Hyper-V Server 2012 R2:

  1. Live Migration Compression, and
  2. Live Migration via SMB Direct (RDMA)

Let’s talk about those, shall we?

Live Migration With Compression

Did you know that your hypervisor host isn’t typically suffering much when it comes to processor capacity?

“I didn’t know that.”

It’s true.  So, what we’re going to is borrow some extra CPU cycles while we’re doing a live migration, and actually compress the migration data before it goes over the wire, and decompress at the destination. 

If it sounds just that simple, well, it is.  And it’s just a simple choice in the Live Migrations –> Advanced Features settings on your Hyper-V hosts:

image

And as if that wasn’t good enough…

Live Migration via SMB Direct (RDMA)

In Windows Server 2012 we introduced a new version of SMB – SMB 3.  Among other things, this version of the protocol greatly improves performance; even to the extent that we can trust a basic file share to be the location for live data such as a virtual machine’s hard disks and data disks, or a SQL Server database.  (Click here for a good summary of what SMB 3 provides.)

SMB Direct (SMB over Remote Direct Memory Access, or RDMA) is technology that, given hardware (the NICs) supporting it, can establish an efficient memory-to-memory transfer of data.  In Server 2012 the main beneficiary of this was faster file services.  But in R2 we’re using this to send live migration data between the Hyper-V hosts. 

image

So now instead of just sending the memory and configuration of a VM over the wire using TCP/IP, or compressing it first, we’ll use a direct memory-to-memory channel. 

Can you say “FAST”?

“Fast!”

I knew you could. 

“But, can you give me an example?  Can you show me how they compare?”

The best example I can give you is Jeff Woolsey’s demonstration he did for the TechEd 2013 North America keynote this past June. 

Click this link to watch his demo (at 1:56:15) : TechEd 2013 North America Keynote Video – Jeff Woolsey’s Live Migration Demo

Click to watch Jeff Woolsey's demo.

And for a more detailed description of Live Migration and the improvements made, check out this page: Virtual Machine Live Migration Overview

Questions?  Comments?  Make sure you add them to the comments at the bottom of this post!  And try it out yourself by downloading the evaluations of either Windows Server 2012 R2 or Hyper-V Server 2012 R2

---

**That’s a ‘vMotion’ for those of you who are more familiar with the VMware terminology.

  • Two days of in depth talks on Hyper-V in San Francisco November 14-15 by the IT Pro community.  More info at www.techdays.org

  • Thanks, Doug.  Is Jennelle involved in this?  I should hope so.  :)

  • Yes she is.  Hyper-V is a very hot topic for IT Pros right now and we are seeing lots of jobs for folks who know Hyper-V.  These two days are meant to give IT Pros the latest in depth info on Hyper-V, VMware, High Availability and all the new storage options.

  • Fantastic!  I wish I could join you guys there again.  Say 'hi' for me.  :)

Blog - Post Feedback Form(CAPTCHA)
Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment