Creating HA VMs for Hyper-V with Failover Clustering using FREE Microsoft iSCSI Target 3.3

In my last piece (Creating a Storage Area Network Using Microsoft iSCSI Software Target 3.3) I showed you how to create software Logical Unit Numbers (LUN). In this article we are going to take what we have learned to leverage the benefits of a SAN device to create redundancy for our Hyper-V environment. We will:

1. Create a failover cluster between two servers (nodes);
2. Enable Cluster Shared Volume (CSV);
3. Ensure Hyper-V is properly configured on both nodes;
4. Create a highly available virtual machine (HAVM); and
5. Test both Live Migration and Failover between the nodes.

The equipment I am using for these demos are matching HP EliteBook laptops with Intel Core2 Duo T7700 CPUs and 8 GB of RAM. They are both running Windows Server 2008 R2 Enterprise Edition Service Pack 1. For most clustering there is no requirement for identical machines. For Hyper-V Live Migration, the ultimate goal of the article, the CPUs of all nodes have to be of the same family (a limitation of live migration from any vendor). Enterprise Edition (or Datacenter Edition) is required as Failover Clustering is not a feature of Windows Server Standard Edition. Because we are also using Cluster Shared Volumes this must be performed on Server 2008 R2.

Proper networking is of course a requirement… although I strongly recommend having a dedicated network for storage, in this case I am using a simple D-Link gigabit switch to connect my two portable nodes, and have ensured that proper networking has been configured between my hosts:

• host1.alpineskihouse.com (172.16.10.10)
• host2.alpineskihouse.com (172.16.10.11)

Connecting your second server to the iSCSI Target

In the previous article we configured our LUN to present to host1. Before we create our cluster we have to repeat these steps for host2, or there will be no shared storage.

In the server where the iSCSI Software Target is configured:

1. Open the iSCSI Target (Start – Administrative Tools – Microsoft iSCSI Software Target)
2. Right-click on LUN1 and click Properties.
3. In the iSCSI Initiators tab click Add…
4. In the Add/Edit Identifier screen select IP Address as the Identifier Type. Enter the Value as the IP Address of your second server, and click OK.

A dialogue box should appear warning you that you are assigning multiple initiators to a single iSCSI target. Accept the warning by clicking Yes. Click OK in LUN1 Properties.

In the second server (host2)

1. Open the iSCSI Initiator (Start – Administrative Tools – iSCSI Initiator (If there is a notice to enable the iSCSI Initiator Service accept it)
2. In the Targets tab enter the IP address of your iSCSI Software Target and click QuickConnect…
3. In the Quick Connect window you should have confirmation that you have discovered a target, along with its IQN. Click Done.
4. In the Volumes and Devices tab confirm that your two volumes are listed, and click OK.
5. Open Disk Management, and bring both volumes on-line. Assign them the same drive letters you had assigned them in host1.
6. Log off of host2.

Create a failover cluster between two servers (nodes)

The first step requires installing the Failover Clustering feature in Windows Server. As with any other feature this is fairly simple:

1. In Server Manager click on Features, then click Add Features.
2. Select Failover Clustering and click Next.
3. On the Confirm Installation Selections screen click Install.
4. When you have successfully installed the feature, repeat the process on all servers.
5. In Host1 open Failover Cluster Manager (FCM) (Start – Administrative Tools – Failover Cluster Manager)
6. In FCM click Validate a Configuration…
7. In the Select Servers or a Cluster screen enter the names of the servers (in this case Host1 and Host2, which can be added individually or together separated by a semi-colon) and click Add; when the servers appear in the Selected Servers box click Next
   
8. In the Testing Options screen, make sure that radio button next to Run all tests (recommended) is selected.
   
   Note: It is an easy step to skip… and a potentially disastrous one. Running the Validation tests lets you see what might not work properly so that you can fix it. By not running the Validation tests you can not only set yourself up to fail, you can also pretty much guarantee that when your cluster does fail that nobody will be able to help you! 
   
9. On the Confirmation screen click Next to begin the tests. This process will run for several minutes. Be patient!
10. Once the Validation tests are complete you should receive a message in the Summary screen that looks like the one below, stating that your configuration appears suitable for clustering, despite some warnings. You can view the complete report if you’re curious. Once you are finished, click on the link to Create the cluster now using the validated nodes….
    
    The Create Cluster Wizard will open. After reading the information click Next on the Before You Begin screen.
11. In the Access Point for Administering the Cluster screen assign the cluster a name and an IP address in the appropriate subnet, and click Next.
12. On the Confirmation screen confirm your settings are correct and click Next.

The cluster will take a few minutes to create; be patient! You should get a Summary screen like the one below. Click Finish.

Enable Cluster Shared Volumes

Cluster Shared Volumes (CSVs) is a feature of Failover Clustering in Server 2008 R2 for Hyper-V; it is a standard clustered NTFS volume that is available to all nodes that gives the HAVM (Highly Available Virtual Machines) complete mobility as any node can be an owner.

To enable CSVs:

1. On host1 open Failover Cluster Manager (Start – Administrative Tools – Failover Cluster Manager).
2. If your cluster is not opened automatically, click Manage a Cluster and select your cluster from the drop-down list.
3. In the main window under Configure click Enable Cluster Shared Volumes… You should receive the following warning in a pop-up window:

“The Cluster Shared Volumes feature is only supported for use with Windows Server 2008 R2 Hyper-V role. Creation, reproduction and storage of files on Cluster Shared Volumes that were not created by the Hyper-V role, including any user or application data stored under the ClusterStorage directory of the system drive on every node, are not supported and may result in unpredictable behavior, including data corruption or data loss on these shared volumes.

For information regarding support services, please see http://go.microsoft.com/fwlink/?LinkId=137158.”

Click the check-box and click OK. Cluster Shared Volumes should now appear in the navigation pane under your cluster.

1. In the navigation pane click on Cluster Shared Volumes.
2. In the Actions Pane click Add storage.
3. The Add Storage window will appear; select the Cluster Disk 2. You can expand it to see your configuration is correct (as shown). Click OK.

Under Cluster Shared Storage you should now have your CSV configured. It is essentially a hard-link to your VHD that sits on the C drive on the host node under C:\ClusterStorage\Volume1.

Ensure Hyper-V is properly configured on both nodes

Ensure the Hyper-V role is installed on all nodes in the cluster

Hyper-V is an easy role to install. To ensure that it is installed, open Server Manager on each node, click on Roles, and ensure that Hyper-V is listed under Roles Summary. If it is not, add it now.

Ensure Virtual Networking is properly configured on all nodes in the cluster

Under Virtual Networking you must ensure that each host has a virtual network that is identically configured. The name and connection type must match, else failing over will truly fail.

Create a Highly Available Virtual Machine (HAVM)

There are two ways to create a highly available virtual machine.

1. Configure an existing Virtual Machine (VM) as Highly Available; or
2. Create a new Highly Available Virtual Machine (HAVM).

For the purpose of this article we are going to create an HAVM.

1. In host1 open Failover Cluster Manager (Start – Administrative Tools – Failover Cluster Manager).
2. In the navigation pane navigate to Ski-Cluster – Services and applications.
3. In the actions pane click Virtual Machines… click New virtual machine >. Select the node on which you wish to initially create it. For the sake of this article I will create it on host2.

The New Virtual Machine Wizard will launch. Create the VM as you would any VM, ensuring that:

• The Location of the VM is C:\ClusterStorage\Volume1\.
• The Connection is connected to a virtual network that is identically configured on both (all) nodes.
• The virtual hard drive is created in C:\ClusterStorage\Volume1\. If you are using a pre-created virtual hard disk, first move it to the shared storage.
• Install an OS on the new VM.

Note: You can install the OS on the VM on host2 from host1 in a number of ways; one way is to open Hyper-V Manager in host1, right-click on Hyper-V Manager in the navigation pane, and click Connect to Server… then type the host name (or IP Address) of host2. However you can also launch the Virtual Machine Connection from within the Services and Applications window in Failover Cluster Manager.

Configuring HA and Testing Live Migration

Once you have an HAVM created, we can configure it and then test it out.

1. In host1, open Failover Cluster Manager and Hyper-V Manager. Connect to host2 and confirm that your highly available virtual machine is running.
   
2. In Failover Cluster Manager navigate to Services and applications. Right-click on your HAVM and click Properties.
3. In the General tab select Host2 as the Preferred owner. Ensure the checkboxes for Enable persistent mode and auto start are selected.
4. In the Failover tab select the radio button Allow failback. Click OK.
5. In the Services and applications window right-click on your HAVM; hover on Live migrate virtual machine to another node and click (in the call-out) 1 – Live migrate to node Host1.

The HAVM should migrate relatively quickly to Host1. Verify in the Services and applications section that your HAVM is now owned by Host1. In Hyper-V Manager manage Host1 and verify that it is there.

Following the same steps as above, Live Migrate your HAVM back to Host2.

Testing HA Failover

Now that we have performed an Active Live Migration, we have three tests left to go:

• Perform a shutdown Live Migration
• Verify Failback
• Test Dead-hardware Failover.

1. On Host1 make sure Failover Cluster Manager is open and that you can see that your HAVM is running on Host2.
2. Log into Host2 and Restart it!

Notice in Failover Cluster Manager that the HAVM is restored on Host1 but in reality it is a Quick Migration, and the VM is not rebooted.

Once Host2 is rebooted, after a few minutes your HAVM should fail back to its preferred owner. This may take a few minutes! Do not continue until the current owner for your HAVM is Host2.

Our last test is so simple that you can ask the least technical person in the room to do it. Turn off Host2. Do not perform a shutdown – this is supposed to simulate hardware failure. If it is a laptop unplug the power source and then remove the battery. If it is a server then just yank the power cord!

Within a second or two the HAVM should restart on Host1… it will be the equivalent of a dirty boot, but it will come up, alive and well. When you restart Host2 the HAVM will fail back to it just as it did before.

Does it look and feel like black magic? Probably… but you’ll get used to it; you will get used to high availability without the high costs; you will get used to redundancy without excess hardware. Most of all you will be able to rest easy, knowing that your Hyper-V environment is truly highly available, and that you were able to do it all using a bunch of free tools from Microsoft.

What’s next? Build it! Test it! Implement it! Love it! At long last we have true redundancy available for all.

Mitch Garvis, MCT | Senior IT Consultant & Trainer | SWMI Consulting Group
My blog | Twitter | Facebook | LinkedIn | MVP Profile

I'm a PC