Hi—I am Rob Emanuel from the Microsoft.com Operations team. For those of you who may not know what we do, our group designs, deploys, manages and sustains highly available, highly scalable Web and SQL systems for Microsoft for some of the largest corporate web sites in the world (www.microsoft.com, Microsoft Update, Download Center, MSDN and TechNet). Along with our team’s TechCenter, we maintain a blog on how we adopt our own products and share some of our real world experiences.
For the last several months we have had the opportunity to focus on virtualizing both the MSDN and TechNet websites with Windows Server 2008 Hyper-V as a start to our overall virtualization adoption strategy. This was a group effort across our entire Operations Team including individuals from the Technical Architecture group I am part of, the System Engineers who run the sites, the data center hosting team which handles our infrastructure changes and the very supportive product group which is responsible for MSDN and TechNet. Today we are very pleased to be able to share how Hyper-V was deployed for those two sites and our lessons learned through that process. We have written an article on the TechCenter which goes through how we approached virtualizing MSDN and TechNet and hopefully conveys how successful we found Hyper-V to be as a web platform.
The article covers the reasons and characteristics which made MSDN the first site we looked to move onto Hyper-V. It provides an overview of how both Hyper-V Beta and Hyper-V RC0 were deployed as well as the general architecture used for the deployment. Possibly the most surprising finding was that Hyper-V was far more stable than we had expected for a beta version deployment. There was in fact no difference we found in stability or availability between Hyper-V and a physical deployment! We were also not able to identify any bugs for the Hyper-V team during our deployment under either full production load or even stress load.
An excerpt from the article:
Implementation
The rollout of Hyper-V in our production environment followed our standard approach to new technology adoption. Our architecture allows us to use global and local load balancing to quickly bring clusters or individual servers into and out of rotation. We heavily leverage this agility during our technology adoption efforts. This helps us safely bring new systems under production load in a controlled manner, while ensuring a continued positive customer experience.
After analyzing the results of some standard performance tests (for example, memory, CPU, and I/O) our confidence in Hyper-V was such that we moved to site-specific testing in production. We monitored the site with normal production monitors while gathering detailed performance metrics specific to each physical server and virtual machine (VM). The production monitors include basic server health as well as end-to-end application tests. This ensured that the deployment was meeting or exceeding previous MSDN and TechNet site performance and availability targets.
Hyper-V Beta Deployment
Our production testing began in early February 2008, when we installed the Hyper-V role on two physical servers, with each hosting three VMs running MSDN. Production load on these six VMs progressed from a cautious 1 percent to 20 percent very quickly and smoothly. During the next six weeks, we tested various amounts of load and VM combinations to better understand the performance characteristics and scalability of the product and the site. MSDN was also deployed directly onto matching physical servers to compare VMs against physical performance, scale, and stability with the same load characteristics.
In an effort to push the site and Hyper-V to their performance limits, we replayed production IIS logs by using the Web Capacity Analysis Tool (WCAT) to understand the upper range performance and scale characteristics.
We tested and compared one, two, and finally three VMs per physical server. The data gathered is in line with those discussed in the "MSDN and TechNet Virtualization Results" section later in this article. The performance and stability of MSDN on the Hyper-V Beta release were so encouraging that, with RC0 nearing release, we began making preparations to move to the next phase of implementation.
Update: The full article can be seen here.
The success of Hyper-V as a web platform for both MSDN and TechNet for performance and stability has given us the confidence to accelerate our plans to implement Hyper-V for many of the other sites we run, such as the live traffic testing of www.microsoft.com on Hyper-V, which we have already started. We are also actively working on our internal adoption of SCVMM 2008 as a major part of our overall virtualization strategy.
As we deploy more systems with Hyper-V and have information from our adoption of SCVMM 2008 we will share that on our TechCenter in the form of additional articles or blog entries.
Happy deployments-
Rob
