David Ziembicki on Infrastructure Architecture : Datacenters

Private Clouds: Believe it or not?

davidzi — Sun, 23 Aug 2009 23:01:01 GMT

Recently there has been some healthy debate around the validity of “private clouds” and whether such a construct is new or just a different name for virtualization and automated provisioning, i.e. a marketecture. In this corner for the anti-Private Cloud sentiment, we have the 2009 Prediction – Rise and Fall of the Private Cloud which argues there is really no such thing as a private cloud and the concept will die quickly as everyone moves to the public cloud. There are some excellent points in this article, I’ll address several below. Take a quick read and come back, I’ll wait… In the other corner, we have Christofer Hoff from Cisco and his response. Finally, Reuven Cohen hits the nail on the head with his post A Public Cloud by Any Other Name is Private which basically states that this is all basically definitional dancing where people argue about concepts without defining any of the underpinning terms.

People’s opinions on this topic seem to correlate most with whether they believe the defining attributes of cloud computing are financial (only paying for capacity utilized, no capex, etc) or whether they believe the defining attributes are technical (shared infrastructure, scale-out architectures, dynamic provisioning). The folks who focus on the financial side tend to believe either there cannot be private clouds because all costs are still incurred by the organization or that there is no way a single organization will be able to drive costs as low as the large cloud providers can with economies of scale.

The folks who see cloud computing as more of an architecture pattern for applications and an infrastructure/operational model tend to believe that the approach is just as relevant for a public cloud provider as it is for a large internal IT organization.

I am firmly in the camp of those who believe private clouds are going to be an important part of IT for at least the next decade. I come to this view using my definition of a cloud which is: an infrastructure architecture, application development model, and operations management discipline that dynamically provide necessary services whenever and wherever they are needed while sharing costs between all users.

Using that definition, all manner of clouds including public, private, hybrid, etc. will exist. Will there be certain economies of scale that a Microsoft or Amazon with hundreds of thousands of servers will be able to achieve that a single business won’t? Of course. But there will also be a degree of customization and agility that private clouds will be able to achieve that large providers won’t.

The reason I believe that private clouds as a concept are something new is that this is the first time that all of IT (infrastructure, development, and operations) are being looked at holistically. This is much more than just being able to sling VMs about the datacenter. This about providing a cost effective infrastructure where code that addresses user needs, be it an app, a VM, or a service can be developed rapidly by using foundational services, deployed near real-time, scale as needed, and then be retired at the end of its useful life.

I’m excited by Microsoft’s opportunities along the full spectrum of the cloud. Azure is a very forward looking vision of the public cloud that I still don’t think most people are grasping yet. Likewise, Microsoft’s traditional on premise solutions are evolving very quickly toward both private cloud and public cloud implementations. To me the most important question that will determine our long term success is how well we are able to provide a seamless continuum between the Azure platform and our Server and Tools solutions as they evolve toward cloud services. I think we are targeting an end game that no one else is really going after from on-premise, through private cloud, to public cloud.

Cloud Computing: Taxes Analogous to Atmospheric Pressure

davidzi — Thu, 06 Aug 2009 16:11:23 GMT

In a blog posting earlier this week, the Azure team announced that they would be moving all Azure applications out of our “USA – Northwest” datacenter. I was fascinated by this given that the stated reason for this move is a change in local tax law which presumably make it less financially attractive to offer the services from that area. Mike Manos published a great blog post on the topic this morning called “The Cloud Politic – How Regulation, Taxes, and National Borders are shaping the infrastructure of the cloud”. Definitely worth reading and considering the implications.

So where will cloud infrastructure form? Consider the real thing in nature and substitute taxes for atmospheric pressure. Below is a paraphrased description from NOAA:

Wind is simply the air in motion … Pressure gradient is the difference in pressure between high and low pressure areas … What happens to the converging winds near a low? … It has to go somewhere so it is forced to rise. As it rises it cools. When air cools it can hold less water vapor so some of the invisible vapor condenses, forming clouds and precipitation … What about the diverging air near a high? … As air warms it can hold more water vapor, which means that clouds will tend to evaporate.

Bottom line, cloud infrastructure will tend to emerge in low tax, low energy cost, high connectivity areas. This much is obvious and has been a key part of data center site selection methodologies as Manos alluded to. To date these have mostly dealt with “where do we plant these multi-hundred million dollar facilities to exist for at least 10 years”. As the move by the Azure team demonstrates however, what runs in these datacenters can be moved around at will. Is it running internal applications and thus maybe not taxable activity? Or is it running a revenue generating activity that may be taxed? If so, does the datacenter in the next state provide a lower tax environment? If yes, move the workload there, and so on.

With an ever growing percentage of computing likely to migrate toward the large cloud providers, small percent differences in the tax rate, cost of power, etc. can have a large impact on the profitability of providing cloud services. You see this today with certain localities actively shaping public policy around attracting datacenter construction.

Over time I think this will lead to several architectural trends. The first is that an ever increasing number of input parameters (tax rate, power cost, bandwidth, etc) will be utilized by cloud infrastructure software to determine where best to run customer workloads. Where today this occurs mostly during site selection, this will rapidly evolve to the point where it is near real-time and workloads will transparently migrate to follow low cost off-peak power, regions with lower taxes, etc. While workloads are easier to move than entire datacenters, even that is very likely to change given the numbers at stake. Most people have heard of Microsoft’s Chicago datacenter where the first floor is comprised of shipping containers and totals hundreds of thousands of servers. This capacity is obviously mobile but requires supporting facility infrastructure which to date is in fixed locations only. If you look at Microsoft’s Gen4 datacenter vision, you’ll see that eventually even most of the supporting infrastructure will be modular and mobile as well.

These trends will make for some very interesting infrastructure architecture challenges. The clouds will form near low pressure areas…

Head-to-Head: Workflow Studio vs PowerShell for Automation

davidzi — Sun, 28 Jun 2009 21:07:11 GMT

Here’s an interesting and slightly amusing mock debate between Brandon Shell and Jason Conger on Citrix’s Workflow Studio vs PowerShell for automation. If you aren’t familiar with it, here is the description of what Workflow Studio is:

“Citrix Workflow Studio™ is an infrastructure process automation platform that enables you to transform your datacenter into a dynamic delivery center.”

“Built on top of Windows PowerShell™ and Windows Workflow Foundation, Workflow Studio provides an easy-to-use, graphical interface for workflow composition that virtually eliminates scripting. Workflow Studio acts as the glue across the IT infrastructure allowing administrators to easily tie technology components together via workflows.”

The debate is amusing because in reality both guys understand that each has its place, one is a foundational component of the other, and the combination of the two can be extremely powerful. The core of the “debate” is one’s definition of automation: execution of atomic tasks with as little effort/code as possible (basic PowerShell) or event/workflow driven execution of multiple tasks with associated logic (advanced PowerShell and/or Workflow Studio). The first is an enabler for the latter.

It’s been my opinion since Exchange 2007 and Virtual Machine Manager 2007 committed entirely to PowerShell and with the PowerShell team’s continued focus on simplicity and consistency, that this was the tipping point that was going to enable real automation and orchestration of IT infrastructures. Now with partners (Citrix) and competitors (VMware) alike building on and/or leveraging PowerShell, we’re going to see significant advancements in the state of the art this year.

Threat Modeling Guide for IT Infrastructure

davidzi — Tue, 23 Jun 2009 02:41:07 GMT

Over on the Solution Accelerators Security Blog is a post and link to the IT Infrastructure Threat Modeling Guide.

From the guide:

The IT Infrastructure Threat Modeling Guide provides an easy-to-understand method for developing threat models that can help prioritize investments in IT infrastructure security. This guide describes and considers the extensive methodology that exists for Microsoft Security Development Lifecycle (SDL) threat modeling and uses it to establish a threat modeling process for IT infrastructure.

This is one example of what I think will be a growing trend where the lines between infrastructure and development will be blurred. This is a positive as there are a substantial number of best practices in both disciplines that can be shared. A structured approach to threat modeling is a prime example.

Excellent Data Center Article in the NY Times Magazine

davidzi — Thu, 11 Jun 2009 20:46:00 GMT

The New York Times magazine has a very interesting article by Tom Vanderbilt on datacenters and the infrastructures that power today’s online services. Several Microsoft (and former Microsoft) execs were interviewed including Mike Manos, Debra Chrapaty, Daniel Costello, etc.

The article ends with a very insightful comment by Mike Manos:

“We’re at the beginning of the information utility,” he said. “The past is big monolithic buildings. The future looks more like a substation — the data center represents the information substation of tomorrow.”

White Paper: Comparing the Power Utilization of Native and Virtual Exchange Environments

davidzi — Wed, 10 Jun 2009 15:07:12 GMT

The Exchange team has published a whitepaper on TechNet comparing the power utilization of a typical 8 server physical Exchange environment compared to a a virtualized environment using the same number of logical processors but on only two Hyper-V host servers. The result: a 50% reduction in power usage (excluding storage). If storage is included it was a 37% reduction.

The purpose of the test was to isolate on power consumption scenarios so other things like cooling and space savings weren’t considered but would also be measurable benefits. There is also a link to general guidance on virtualizing Exchange as it makes sense in a lot of scenarios but not all.

Hyper-V and VMM: Scale Up, Out, and to the Cloud

davidzi — Fri, 15 May 2009 00:44:32 GMT

Between MMS and TechEd there have been a lot of announcements on the virtualization and cloud computing front. First, over on the Virtualization Team Blog, Jeff provided the announcement and details around some new capabilities coming in Hyper-V with Windows Server 2008 R2:

64 logical processor support. A 4x improvement over Hyper-V R1 and means that Hyper-V can take advantage of larger scale-up systems with greater amount of compute resources.

Support for up to 384 Concurrently Running Virtual Machines & 512 Virtual Processors PER SERVER. We are increasing the maximum number of concurrently running virtual machines to 384 per server and the maximum number of virtual processors to 512 for the highest virtual machine density on the market.

Processor Compatibility. Processor compatibility allows you to move a virtual machine up and down multiple processor generations from the same vendor. This does not mean you can live migrate between Intel and AMD nodes, just between different generations from the same vendor.

Not to be outdone, the VMM team announced a bunch of new features that will be in their Release Candidate coming out in a few weeks:

Storage Migration
Queuing of Live migrations
Rapid Provisioning
Host compatibility checks
Support for 3rd party CFS
Support for Veritas Volume Manager

Combined, these new features from both teams enable some key scenarios at both the entry level and high end of the spectrum. One of the major advantages of our stack is that it is very approachable from an entry level since it leverages so much of what your administrators already know and beginning with Microsoft Hyper-V Server 2008 R2 SKU, will be available with all of the high end features (Clustering, Live Migration, etc) for free. Within a couple hours a Windows admin can become proficient with the basics of Hyper-V and be up and running (for free!). Within a few days at most, the ability to implement basic clustering, HA, and Quick/Live migration can be achieved. At the high end, very advanced architectures can be implemented including VMM, OpsMgr, deep SAN integration, etc. This is where our technical guidance, solution accelerators, and service offerings come into play.

To see an example both of how this stack is being leveraged by commercial providers as well as an example use case for enterprises wishing to use the cloud as reserve capacity, check out the video below demoing a future version of VMM and how it will integrate private and public cloud capacity seamlessly:

One Million Server Datacenter?

davidzi — Wed, 29 Apr 2009 17:16:34 GMT

IEEE Spectrum has an interesting article online about mega-datacenters and whether a million server datacenter is possible in the near future. It describes the efforts of the major players in this space. They describe some of the attributes of Microsoft’s Quincy and Chicago datacenters as well as several others.

Last year I had the opportunity to tour one of Microsoft’s smaller datacenters (“only” 45,000 servers) with a couple of our customers. The scale of the facilities and supporting infrastructure are pretty amazing. The things that impressed me most was the attention to detail and discipline required to operate such a facility. In walking through for a couple hours I didn’t see a single wire out of place, everything was pristine. The datacenter manager hosting the tour made a point that really struck me about the discipline required. He said that even the smallest non-compliance with facility rules and standards (i.e.. tools left behind, improper cabling, etc) resulted in disciplinary action. He said that when operating such a large and complex facility that attention to detail is critical else things can quickly start to spin out of control. Another interesting point was that only a small double digit number of people are required to staff the facility in total across three shifts.

The IEEE article talks primarily about the current Microsoft datacenters. For some insights on where we are going in the future, the following articles and video may be of interest.

Our Vision for Generation 4 Modular Data Centers - One way of Getting it just right

Microsoft's Generation 4 Data Center Vision - the Architects' Perspective

Designing Generation 4.0 Data Centers: The Engineers’ Approach to Solving Business Challenges