Kevin Remde's IT Pro Weblog
IT Pro Resources
TechNet EventsMicrosoft Security Response CenterTechNet IT Manager Community HubMicrosoft Virtual AcademyKevin’s Evaluation Download Center
IT Pro Evangelist Blogs
Blain Barton Blain Barton's Blog@BlainBar
Brian LewisMy Thoughts on IT...@BrianLewis_
Dan Stolts IT Pro Guru Blog@ITProGuru
Jennelle Crothers TechBunny@jkc137
Keith MayerIT Pros ROCK!@KeithMayer
Kevin Remde Full of I.T.@KevinRemde
Matt Hester Matthew Hester's WebLog@MatthewHester
Tommy PattersonVirtually Cloud 9@Tommy_Patterson
Yung Chou Yung Chou on Hybrid Cloud@YungChou
Okay.. I feel like sharing this because it’s pretty stupid, but in a geeky-sort-of-way the solution was interesting enough to share. Think Chicken & Egg. (or “Catch-22”).
As the title of this post suggests, the subject is Windows Failover Clustering. For those of you who are not familiar with it, Windows Failover Clustering is a built-in feature available in Windows Server 2008 R2 Enterprise and Datacenter editions. Along with shared storage (for which we used the free iSCSI Software Target from Microsoft to implement), it provides a very easy-to-configure and use cluster for serving up highly available services. In our case, this would be virtual machines running on two clustered virtualization hosts.
As a training platform, but primarily for use as a demonstration platform for our presentations (and certainly more real-world than one laptop alone can demonstrate), our team received budget to acquire several Dell servers. We found a partner (Thank you Groupware Technology!) who was willing to house the servers for us. The idea was that we, the 12 IT Pro Evangelists (ITEs) in the US would travel to San Jose in groups of 3-4 and do the installation of a solid private cloud platform, using Microsoft’s current set of products (Windows Server 2008 R2 and System Center). This past week I was fortunate enough to be a member of the first wave, along with my good buddies Harold Wong, Chris Henley, and John Weston. The goal was to build it, document it, and then hand if off to the next groups to use our documentation and start from scratch, eventually leaving us with great documentation, and a platform to do demonstrations of Microsoft’s current and future management suites.
We all arrived in San Jose Monday morning, and installed all 5 server operating systems in the afternoon. We installed them again Tuesday morning.
It’s a long story involving how Dell had configured the storage we ordered. We needed to swap some drives between machines and set up RAID and partitioning in a way that was more workable to our goals. I’ll leave that discussion for one of my teammates to blog about.
Anyway, once we had the servers up, I installed and configured the Microsoft iSCSI software target on our “VMSTORAGE” server, and configured two other servers as Hyper-V Hosts in a host cluster, with Windows Failover Clustering and CSV storage. By the end of the week we had overcome hardware, networking, missed-BIOS-checkmarks (did you know that Hyper-V will install, but you can’t actually use it if you somehow miss enabling Virtualization support on the CPU on one of the host cluster machines? Who’da thunk it?!) , we had 5 physical and a half-dozen virtual servers installed and running, with Live Migration enabled for the VMs in the cluster. Our domain had two domain controllers; one as a clustered, highly-available VM, and the other as a VM that was not-clustered, but still living in the CSV volume; C:\ClusterStorage\Volume1 in our case. (That’s a hint, by the way. Do you see the problem yet?)
One of the many hurdles we had to overcome early on was an inadequate network switch for our storage network. 100Mbps wasn’t going to cut it, so until our Gig-Ethernet switch arrived on Friday, Harold used his personal switch that he carries with him. On Friday before we left for the airport, we shut down the servers and let the folks there install the new switch. Harold need his switch back at home.
But in restarting the servers, here’s the catch: Windows Failover Clustering requires Active Directory. The storage mount-point (C:\ClusterStorage\Volume1) on our cluster nodes requires the Failover Clustering. And remember where I said our domain controllers were?
“Um.. So… Your DCs couldn’t start, because their location wasn’t available. And their location wasn’t available, because the DC’s hadn’t started. And your DC’s couldn’t start, because their storage location wasn’t available, and… !!”
Bingo. Exactly. Chicken, meet Egg. It was our, “Oh shoot!” moment. (Not exactly what I said, but you get the idea.)
“So how did you fix it?”
I’ll tell you…
Our KVM was a Belkin unit that supports IP connections and access to the machines through a browser. We configured it to be externally accessible. So I was able to use that to get in to the physical servers and try to solve this “missing DCs” puzzle; though to make matters much more difficult, the web interface for that KVM is really, REALLY horrible. The mouse didn’t track to my mouse directly, no ALT+ key support, TAB key didn’t work.. I ended up doing a lot of the work from a command-line simply because it was easier than trying to line up and click on things! Perhaps in a future blog post I will give Belkin a piece of my mind regarding this piece-of-“shoot” device…
So, my solution was based on two important facts:
“Ah ha! So on the storage machine, you mounted the .VHD file that was your cluster storage disk, and you copied out the .VHD file from one of the domain controller VMs!”
Yeah.. that’s basically it. Though I did have one problem. The .VHD file was in-use; probably by the iSCSI Software Target service. So when I tried to attach it, the OS wouldn’t let me.
Fortunately I found that by stopping that “Microsoft iSCSI Software Target” service on the storage server (I also stopped the “Cluster Service” on the two Hyper-V cluster nodes), I was able to attach to the .VHD, navigate into it, and copy out the .VHD disk for the needed Domain Controller. (Actually, I also removed the .VHD from its original location. I didn’t want the DC to come alive again when the storage came back online, if the identical DC was already awake and functioning.)
So after that, it was as simple (?) as this:
Everything came back to life almost immediately; including the Remote Desktop Gateway that we had configured so that we could remotely connect to the machines in a more meaningful, functional way.
So the moral of the story is: When you’re building your own test lab, or even considering where to put your DCs in your production environment, make sure you have at least one DC that comes online without depending upon other services (such as high-availability solutions) that, in turn, require a DC to be functioning.
All-in-all, it was a great week.
Do you have any similar stories? Share them with us in the comments. We’d love to hear ‘em!
We IT Pro Evangelists are often asked, “Hey.. Where can I find the download to try out such-and-such software?”
And after I find out what they actually meant by “such-and-such”, I usually just tell them essentially to “google it on Bing”.
But that’s not sufficient.
So what I am doing instead, on this page, is to give you a list of the most important, most commonly requested downloads and evaluations. And here are the latest and greatest:
We also have evaluations of our recent-yet-older products:
If you're interested in evaluating the newly released System Center 2012 R2 management product, and all of the components therein, you can get them here:
Do you want to try Windows Azure for FREE?
"Yes! Yes I do!"
Here you go:
How about some free online training? We've got literally hundreds of hours of up-to-date training at the Microsoft Virtual Academy:
Try them all!
And if this is not exactly what you need, you can also go to the TechNet Evaluation Download Center and find everything an IT Pro could ever want to try out.
Great news! More betas and release candidates of the System Center 2012 tools are available today for download! Like many of you, I have been waiting patiently for these gems, and I can’t wait to start playing with them. These round out the pre-releases of tools that allow you to support private and hybrid cloud solutions.
System Center Configuration Manager 2012 has already had a couple of betas, so it’s nice to see that it is now up to Release Candidate status. Some of the improvements:
Furthermore, new capabilities released in this RC release include:
System Center Endpoint Protection 2012 is a name that you have never heard before today. That’s because it is an official re-branding of what was Forefront Endpoint Protection. “This name change better reflects the integration between management and security that we have been working towards…” says it pretty well. I think the fact that this solution is not only integrated with, it is “built upon” System Center Configuration Manager is a key part of this, too. It makes sense.
“Sounds great. Which betas are available today?”
The first of the two betas released today is System Center Service Manager 2012, which is, as the name implies, the service management part of the solution.
“You mean, like a help-desk support ticket manager?”
Yes, and much more. I know I’m just scratching the surface of all it can do when I say this – but think about how you will provide a self-service interface for your users and the business units you support. Service Manager is the tool that you will use to create that portal, and then to allow you to act upon requests; and not just manually. Those requests can be handled by you and your staff, or trigger some automation to take care of the request. (See System Center Orchestrator 2012). Also, Service Manager 2012 adds the tracking of SLAs. So if you have a service that you have promised a certain response time or up-time for, Service Manager can track your actual performance against those goals.
For a private cloud, you have to grant people the ability to request and acquire services without having to know the details of the underlying architecture (the “fabric” - Storage, Compute, Networking). Service Manager is one way to provide that interface. More details about the current version, Service Manager 2010, can be found here.
And the other beta available today, formerly known as codename “Concero”, is System Center App Controller 2012.
“Let me guess.. It controls your apps?”
Well.. yeah.. in a sense. More accurately, though; this is the tool that bridges the gap for the management of private and public cloud services. Once the fabric is configured, and the clouds are defined; meaning: once you folks in the datacenter have created the building blocks for the services you provide, and have defined the abstracted layer at which your users can request and manage services (See?! Sometimes it’s just easier to say “cloud”, isn’t it!), or through which you can make changes such as scaling-out your services. For example, adding more instances of the web-tier in a muli-tiered application, for example.
“Couldn’t I just use System Center Virtual Machine Manager to do that?”
Sure, you could – if that person has the VMM 2012 admin console installed. And if all of your clouds are local. But what the application owner doesn’t have that console installed? Or what they also manage multi-tiered applications (or call them “services”) running in some other cloud like Windows Azure?
“Wait.. what? I can manage my Windows Azure-based services with this?”
Yep. As I said, App Controller bridges that gap for you. It’s a self-service portal where the application owners can go to manage their applications that are running in your private and/or public cloud spaces. For those folks, it’s not about managing the infrastructure or the plumbing you’ve provided. For them, “It’s all about the app.”
You can find instructions for installing and configuring System Center App Controller 2012 here.
“So.. the obvious question remains: When will these products be released?”
They are shooting for the first half of calendar year 2012.
DOWNLOAD and try these tools out. But wait just a minute and let me start my download first.
IMPORTANT Software Evaluations – Get ‘em here: http://aka.ms/Evals
"WAG" = Windows Administrator Groups (on LinkedIn) with Subgroups for user groups by geography – http://aka.ms/WAG
Future Live, In-Person Event Information: TechNet Events (for IT Pros): http://www.TechNetEvents.com MSDN Events (for Software Developers): http://www.MSDNEvents.com
Session Slides: http://aka.ms/FY12H1Remde
Microsoft Virtualization: On Your Terms Windows Virtualization Home - http://www.microsoft.com/virtualization
Hyper-V getting Started Guide - http://technet.microsoft.com/en-us/library/cc732470(WS.10).aspx
Step-by-Step Guide to Getting Started with Hyper-V (download) http://www.microsoft.com/downloads/details.aspx?familyid=BCAA9707-0228-4860-B088-DD261CA0C80D&displaylang=en
Case Study: Target Corporation http://www.microsoft.com/casestudies/Windows-Server-2008-R2-Datacenter/Target-Corporation/Large-Retailer-Relies-on-a-Virtual-Solution-to-Deliver-Optimal-Shopping-Experience/4000009407
Case Study: Dartmouth-Hitchcock Medical Center http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000006711
Virtual Machines and Supported Guest Operating Systems http://technet.microsoft.com/en-us/library/cc794868(WS.10).aspx
Microsoft server software and supported virtualization environments http://support.microsoft.com/kb/957006
Bing it: “Managing Hyper-V with WMI” / “Managing Hyper-V with PowerShell” http://www.bing.com/search?q=Managing+Hyper-V+with+WMI http://www.bing.com/search?q=Managing+Hyper-V+with+PowerShell
Hyper-V Security Guide - http://www.microsoft.com/download/en/details.aspx?id=16650
The TechNet Virtualization Scenario Hub - http://technet.microsoft.com/en-us/virtualization/default.aspx
Virtualization Community Resources - http://technet.microsoft.com/en-us/virtualization/cc150662
Virtualization Resources in the TechNet Library - http://technet.microsoft.com/en-ie/library/cc165455(en-us).aspx
Windows Virtualization Team Blog - http://blogs.technet.com/virtualization
VMware Compare: Project Virtual Reality Check - http://www.virtualrealitycheck.net
VMware Compare: Enterprise Strategy Group http://www.enterprisestrategygroup.com/2010/07/microsoft-hyper-v-r2-scalable-native-server-virtualization-for-the-enterprise/ and
“VMLimited” Video - http://www.youtube.com/watch?v=hewedqvSWaI
Microsoft Virtualization: From the Metal Up
The Private Cloud Home Page - http://www.microsoft.com/privatecloud
“Ready Now” Private Cloud Solutions - http://www.microsoft.com/en-us/server-cloud/readynow/default.aspx
The National Institute of Standards and Techology (NIST) Definition of Cloud Computing - http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf
Microsoft’s Definition of Private Cloud - http://www.microsoft.com/virtualization/en/us/private-cloud.aspx
Microsoft Virtual Academy - http://www.microsoftvirtualacademy.com
Microsoft Cloud Home Page – http://www.microsoft.com/cloud
Online Services - http://www.microsoft.com/online/
Microsoft’s Global Foundation Services (The Datacenter People) - http://www.globalfoundationservices.com/
Hyper-V Cloud Deployment Guides - http://www.microsoft.com/virtualization/en/us/private-cloud-get-started.aspx
System Center Virtual Machine Manager 2008 R2 – http://www.microsoft.com/scvmm
SCVMM Self-Service Portal (SSP) 2.0 SP1 - http://go.microsoft.com/fwlink/?LinkId=205827
System Center 2012 Evaluation Download Center - http://www.microsoft.com/click/services/Redirect2.ashx?CR_CC=200069535
SCVMM Team Blog - http://blogs.technet.com/scvmm