If you were at MMS 2008, or if you read executive keynotes in your spare time, you heard Bob Muglia mention that we are running MSDN and TechNet virtualized, in production on Hyper-V.

MSDN and TechNet are full virtualized right now with Hyper-V, and we had a dramatic cost savings when we went and did the virtualization of these very high volume Web sites within Microsoft. Of course, we'll be deploying it throughout our datacenters both for our internal IT as well as for our external facing properties in the coming months.

So how did we do it?  The MSCOM Operations team has just released a White Paper covering the details of the rollout.  From the "Lessons Learned" section:

  • Based on our results with Hyper-V, and a review of current hardware utilization across our environment, we expect to realize significant benefits by consolidating diverse applications on a virtualized platform. Mixing high and low scale systems on the same physical server resources should enable us achieve improved overall hardware utilization and a reduced physical footprint.
  • Although we reduced our hardware supporting MSDN and TechNet by migrating from older physical servers to a smaller deployment of new, more powerful physical servers, there is some minimal overhead associated with virtualization. From our perspective, Hyper-V has clearly delivered enough performance, stability, and scale to drive widespread adoption in our production environment. The flexibility and management gains expected from coupling System Center Virtual Machine Manager (SCVMM) v2 with Hyper-V should justify the measured overhead.
  • If the three percent or less in additional overhead from oversubscribing resources is consistent with additional application workloads tested, oversubscription with Hyper-V should provide MSCOM Ops significant flexibility and reasonable performance for application consolidation.
  • The results we achieved are based on the application characteristics of MSDN and TechNet on Hyper-V RC0. We are working to build a model based on this data that we hope will allow us to predict physical and VM requirements based on common performance characteristics of a Web site. The model will likely include the current requests per second per 1 percent CPU metric with additional memory and I/O performance characteristics as we virtualize other Web applications and gather more test and production data.

image

By the way... if you want to see some crazy diagrams of the overall network infrastructure that powers microsoft.com, check out this site.  Some stats, if you are into that kind of thing:

  • During “Patch Tuesday”, egress can exceed 500 gigabits per second through the Microsoft and CDN partner networks.
  • Averages 56 million unique visits monthly in the U.S.
  • Averages 250 million unique visits monthly worldwide.
  • Generates 70 million page views daily.
  • Averages 15,000 connections per second.
  • Maintains an average of 35,000 concurrent connections to a total of 80 Web servers.