[Updated May’09: Windows Storage Server 2008 now available to MSDN/TechNet subscribers. Checkout Jose Barreto's Blog for details.]

Familiar with Virtual Server 2005 and shared disks for creating virtual clusters?  Well its different with Hyper-V.  The shared disk option is no longer available (which I did not know when I started testing).  You have to use iSCSI instead.  Here is a step by step method for creating a fail-over cluster within Hyper-V.  Its a cheap way of setting up a test lab (assuming you don’t have access to Windows Storage server).  In this post I use StarWind to simulate iSCSI storage … its not an endorsement of the product, I just picked it from amongst the crowd.

Windows Server 2008 fail-over clusters support Serial Attached SCSI (SAS), iSCSI and Fibre Channel disks as storage options.  So, how would you go about setting up a virtual Windows Server 2008 test cluster using the new Hyper-V vitalisation product?  The method I am about to outline is a little different to what you might be used to Virtual Server 2005.  The following steps detail how I managed to setup a test cluster using simulated iSCSI storage.  Before beginning it’s worth reviewing this article that outlines the storage options that are available to Hyper-V.  By the end of this post you should have a simple two node cluster up and running using simulated iSCSI storage.

Tools for the job:

• A Windows Server 2008 server x64 server with the Hyper-V role enabled (I used a Dell Precision 390)
• One Windows Server 2008 VM to act as a Domain Controller (Clusters must be part of a domain) 
• Two Windows Server 2008 VMs to act as Cluster Nodes
• One Windows Server 2003 SP2 VM (or you could use Windows Server 2008 in a Core install to maximise VM performance)
• iSCSI Target Software: I used the StarWind product that is available as a 30 day eval.  Windows Storage Server is now available to MSDN/TechNet subscribers.
• iSCSI Initiator software (built into Windows Server 2008)

I wont go into how to create a VM but you can find more info from Virtual Guys weblog.

Before I began looking into the iSCSI simulated storage option for my cluster nodes I tried to expose a single VHD to each of my cluster nodes in the hopes that they would share it.  I didn’t get very far and was presented with the following error when powering on the VMs:

This error is by design (thanks Justin Zarb for point this out) as Windows Server 2008 Hyper-V does not support this sort of storage (see link above for Hyper-V storage options).  The above error is simply a file system error as the VHD “is being used by another process” … should have spotted that

SETTING UP THE LAB

Note: I’m assuming that you know how to install Windows Server 2003 and 2008.  I’m also assuming that you know how to install and configure a Window Server 2008 Domain Controller.  If you have any questions leave me a comment and I will see if I can point you in the right direction.

VIRTUAL NETWORK

Create the network with a connection type of “Internal Only”.  I enabled Virtual LAN identification and set the default ID to 2 as this will be my public LAN.  Setting the default to 2 means that if I dont specify a VLAN on subsequent NICs they will be classified as public connections.

VLAN ids:

• VLAN 2: Public 10.1.1.x/24
• VLAN 3: Heartbeat 192.168.1.x/24
• VLAN 4: iSCSI 192.168.2.x/24

SERVER SETUP

Tip: Be sure to rename each network card on the hosts to make identification easier.  If its the public NIC, call it public etc.

Domain Controller: dc01

• Windows Server 2008 x32 
• One VHD IDE fixed size disk 10GB
• 1 x NIC connected to my Virtual Network in VLAN 2

Network settings:

• IP Addr: 10.1.1.10
• Mask: 255.255.255.0
• Gateway: I didn’t bother setting one
• DNS: 10.1.1.10

Cluster Nodes:

• Windows Server 2008 x32
• 1 x VHD IDE fixed size disk 10GB
• 3  x NICs connected to my Virtual Network in the following VLANs
  • Public card: VLAN 2
  • Heartbeat card: VLAN3
  • iSCSI: VLAN4

Node01

Public NIC: VLAN 2

• IP Addr: 10.1.1.20
• Mask: 255.255.255.0
• Gateway: I didn’t bother setting one
• DNS: 10.1.1.10

Heartbeat NIC: VLAN 3

• IP Addr: 192.168.1.4
• Mask: 255.255.255.0

iSCSI NIC: VLAN 4

• IP Addr: 192.168.2.4
• Mask: 255.255.255.0

Note: On all NICs in VLAN 3/4 be sure to disable the Client for Microsoft Networks, disable DNS registration and disable NetBIOS.  Be sure to check your binding order too.   The public NIC should be first.

Node02

Public NIC: VLAN 2

• IP Addr: 10.1.1.21
• Mask: 255.255.255.0
• Gateway: I didn’t bother setting one
• DNS: 10.1.1.10

Heartbeat NIC: VLAN 3

• IP Addr: 192.168.1.5
• Mask: 255.255.255.0

iSCSI NIC: VLAN 4

• IP Addr: 192.168.2.5
• Mask: 255.255.255.0

Note: On all NICs in VLAN 3/4 be sure to disable the Client for Microsoft Networks, disable DNS registration and disable NetBIOS.  Be sure to check your binding order too.

iSCSI Target

• Windows Server 2003 SP2 x32 (see here for notes on W2K3 hosts in Hyper-V)
• 1 x VHD IDE fixed sized disk 10GB
• 2 x VHD SCSI fixed sized disks 1GB and 10GB for Cluster disks
• StarWind iSCSI Target Software
• 2 x NICs  connected to my Virtual Network in the following VLANs:
  • Public : VLAN 2
  • iSCSI : VLAN 4

Public NIC: VLAN 2

• IP Addr: 10.1.1.22
• Mask: 255.255.255.0
• Gateway: I didn’t bother setting one
• DNS: 10.1.1.10

iSCSI NIC: VLAN 4

• IP Addr: 192.168.2.2
• Mask: 255.255.255.0

Note: On all NICs in VLAN 3/4 be sure to disable the Client for Microsoft Networks, disable DNS registration and disable NetBIOS.  Be sure to check your binding order too.  Make sure you format and assign drive letters to the SCSI VHDs on this VM.

Setting up the Cluster

Update 17/10/2008: I've also found that using the Image Files option works quite well too.   Image files will allow you to pack more than one VM onto a disk partition.  Check out http://www.starwindsoftware.com/images/content/StarWind_MSCluster2008.pdf for more info.

Note: Check out the how to the same with Windows Storage Server 2003 R2.  http://www.microsoft.com/windowsserversystem/wss2003/productinformation/overview/default.mspx

Update May 09: Windows Storage Server 2008 has now RTM’d and is available online through MSDN and TechNet.  http://www.microsoft.com/windowsserver2008/en/us/WSS08.aspx

Configuring the iSCSI target software (Starwind)

• Install the StarWind software on your iSCSI target VM. 
• Launch the StarWind management console. 
• Under the Connections you should see localhost:3260.  Right click on localhost and select Connect.  If I remember correctly the first username and password becomes the default (which you can change later).

• Right click localhost:3260 and select add Device 
• Select Disk Bridge Device as the Device type and click next

• Select the first SCSI disk from the list (more than likely \\.\PhysicalDisk1). 

• Select Asynronous Mode and Allow multiple iSCSI connections (clustering) and click next 
• Give the disk a friendly name
• Repeat the steps to add the second disk

Adding disks to the cluster nodes

Each cluster node now needs to be connected to the iSCSI target.  Launch the built in iSCSI initiator and follow the steps below:

• If prompted to unblock the Microsoft iSCSI service always click Yes otherwise the 3260 port will be blocked. 
• Click on the Discovery tab and select Add Portal.
• Enter the IP address for the iSCSI target [192.168.2.2]

• Click the Targets tab and you should now see a list of the disks available on the target

• For each disk in the list click Log on and select Automatically restore this connection
• Click on the Volumes and Devices tab and select AutoConfigure.  You disks should now appear as Devices.
• Reboot each cluster node as you add the disks.
• Disks will be offline when you reboot.  Ensure that you bring them online in Disk Management.

When completed (and hosts connected) you should see something like this on the iSCSI target VM.

Installing the Cluster

The new fail-over cluster wizard is quite straight forward and much easier to follow when compared with Windows Server 2003.  There isn't much point in going into too much detail … you’ll find plenty of info on the web.

Here is a step by step guide to installing a two node file cluster in Windows Server 2008.

This is a not so common issue I can across this week . The background is as follows.

• You decided to evict a node from your cluster.
• There is a communication failure between nodes and a warning appears that the cluster was unable to remove clustering components from the evicted node.
• You log onto the node in question and from a command prompt run "cluster node /forcecleanup"

The command responds with the output show in the screen shot below

Your server is now in limbo. You cannot un-install the cluster service and you cannot re-join the cluster, the cluster network driver and cluster disk driver are still online. Most people would re-install the server from scratch at this point. There is however a workaround that will save you time.

1.    Open up regedit on the system in question & Navigate to HKEY Local Machine\Software\Microsoft\Windows NT\Currentversion\Cluster Server

2.   Right click ClusterInstallationState and choose Modify

3.   Change the value to 3

4.   Click OK

5.   Exit Regedit and reboot your server.

6.   Log back into windows and go to a command prompt.

7.   Run "Cluster node /forcecleanup" you should see the following output

That's it !! No rebuild required. You can now operate the node as a stand alone box or join another cluster.

With the economic down turn and the green agenda Virtualisation has become a hot topic with my customer. These days its all about getting the best value for money as possible with your IT budget, so when my customer had a number of servers out of warranty and due for replacement the Hyper-V platform was the first port of call.

The first thing we did was run the Microsoft MAP tool against these servers to ensure that they were real candidates for Virtualisation. This tool can be found at

http://www.microsoft.com/downloads/details.aspx?familyid=67240B76-3148-4E49-943D-4D9EA7F77730&displaylang=en

Information on using the tool can be found at

http://technet.microsoft.com/en-us/library/bb977556.aspx

Currently my customer has a number of Hyper-V GEO Cluster's based on HP boot from SAN Blades. All  of the Virtual Hosts are managed centrally by Microsoft System Centre Virtual Machine Manager (SCVMM). Using the map tool we were able to determine that based on the existing hardware we could achieve an 8 - 1 virtual machine ratio. Considering that the new hardware runs cooler / cheaper and is only a couple of U per blade compared to the 6 - 8 U servers they were replacing everyone was happy.

All of the machines to be Virtualised were Windows 2000 & ran bespoke applications. If we were to rebuild these servers on new kit it would have taken a lot of time and effort to ensure that the applications were tested etc.. not to mention the downtime involved.

Pre-Requisites

You will need the following patches on the Hyper-V target systems.

KB950050, KB951308, KB956589, KB956697, KB956710, KB956774

You will need the following patch on the SCVMM Server

KB959596

You will also need the following version of WAIK for all offline conversions. The version included with the OS will not do the job. Install this on the SCVMM Server.

http://www.microsoft.com/downloads/details.aspx?familyid=C7D4BC6D-15F3-4284-9123-679830D629F2&displaylang=en

Using the P2V Wizard

In this example I am performing a physical to virtual conversion on a Windows 2000 server.

A Windows 2000 server P2V has the following pre-requsites.

• Service Pack 4
• 512MB RAM minimum

As the source service is Windows 2000 the only option is an offline conversion. As part of the process an agent will be installed on the source server and the server will be re-booted into WinPE so that the contents of the source servers hard drive can be copied via BITS.

1. With the Virtual machines menu option high lighted click on Convert Physical Server.

2. Enter the Computer name or IP address of the Physical Server and account details of a user that has local administrator rights on the source Physical Server.

3. Enter a Name for the New Virtual Server. Set the owner of the Virtual Machine (defaults to the logged in user) and add a description for the Virtual Server.

4. Click Scan System to install the SCVMM agent & gather information on the Physical Server.

5. After the scan the System Information panel will be populated. Click Next to continue.

6.  Here we select the volumes to be copied to the new VM as part of the P2V process. You can also change the VHD type from Dynamic to Fixed.

7. Typically on this screen I choose to obtain an IP address automatically from DHCP. You can specify an IP address & Network card (using MAC address) if required.

8.  On this screen you can specify the number of process and amount of RAM the VM will use. I usually set the VM to use 2 processors during the P2V process. This helps with the integration components setup, it can be changed back to a single processor later. Please note that these settings will be used to determine the placing of the VM on a host server as we will see later.

 

9. Here you choose the server that will host the VM. You can see the suitability of each host based on the Star Rating. This is much improved when SCOM is used in conjunction with SCVMM. In the screen shot below SCOM was not configured.

10. As the host I selected was a Windows 2008 Hyper-V cluster I got this message box popping up. Click Yes to continue. SCVMM will set up the new virtual server as a clustered VM.

11. Select the volume that the VM will reside on. If your target volume does not appear on this list refresh the cluster information within SCVMM.

12. Select the Virtual Network that the Virtual Machine will use.

13. As this VM will reside on a cluster do not change these settings. This allows the cluster service to manage the Virtual Machine.

14. If all is ok you should see this screen. You can run into an issue that can occur  with legacy hardware (i.e. RAID controllers) not included with the WINPE Image, which is used to boot into the P2V environment . If you can obtain the Vista driver for the problem hardware copy it to SCVMM\Drivers\Import folder to solve the issue.

 

 

15. This last screen gives us a summery of the P2V job & the option to View and copy the PowerShell script generated by the wizard. You can copy out this script and modify it for automating this process if required.

 

16. After clicking Create the jobs screen will pop up. This screen provides real time information regarding the P2V process including the time required to copy the contents of the targets volumes to the new virtual machine.

 

17. Here we can see the BITS copy in progress and the amount of time remaining to copy the volumes.

18. The script we created with the Wizard also installs the required virtual machine components. In some cases this process will seem to hang. This can be resolved by using the Hyper-V console on the target machine to re-start the VM.

 

20. Once the jobs screen completes we will have a running Windows 2000 VM. Check that the VM is running on the External Virtual Network and that the source machine is turned off and removed from the Network.

The process was quick with minimal downtime for the users. We did run into some issues with legacy RAID controllers but got around them using the fix mentioned in step 14. The new VM’s are a lot more stable than the previous hardware and are now on a high availability platform giving my customer more peace of mind.

I was recently asked a question about PowerShell's ability to read in an XML configuration file at a Virtual Academy I ran last week.  One of the strengths of PowerShell is its ability to perform lots of time saving tasks … one of which is reading in an XML file.  The Get-Content command can read in an XML file and you can easily loop through the contents.

Example:

[xml]$computerlist = Get-Content computers.xml
foreach( $computer in $computerlist.computers.target) 
{ 
    Write-Host $computer.name
} 

What would the XML file look like?

<computers> 
  <target> 
    <Name>server1</Name> 
  </target> 
  <target> 
    <Name>server2</Name> 
  </target> 
</computers>

Nice and simple really.

Where is the cluster log in Windows 2008 ?

This short answer is its no longer there. On our Windows 2008 cluster node if we navigate to %systemroot%\system32\LogFiles\Cluster your wont find the cluster.log file anymore.

Why ? Its been replaced by a much more sophisticated event based tracing system.

The Vista\Windows Server 2008 Event Model is the next generation of Windows Event Logging and replaces the current version of the Event Log shipped in Microsoft® Windows® 2003 Server, Microsoft® Windows® XP, Windows 2000, and previous versions of Microsoft® Windows NT®.

The new model is a major update to the NT Event Log service. It maintains 100% backwards compatibility with the existing APIs and functionality and fully leverages the existing NT Event Log instrumentation in the applications and services. At the same time, it eliminates some of the limitations of the NT Event Log and provides additional features to better support monitoring and diagnostics of Windows applications, services, components, and drivers.

In a future post I will go through the new Logging and tracing features for clusters in Windows 2008 but for now lets look at how to get access to the old familiar cluster.log file.

Here's how to go about it.

1.   Go to a command prompt

2.  Type "Cluster /Cluster:yourclustername log /gen /copy "C:\temp". You should get output as follows

3. Navigate to the c:\temp directory and there you will find the .log files for each node of your cluster.

The cluster log can now be opened in Notepad.

Please note that you need to run this command after each change as its not dynamically updated like the old .log file.