ms datacenters

For an interesting discussion on Windows Live Hotmail and the importance of operating and engineering the service (at huge scale), please visit the Windows Live Team’s Blog site and Arthur de Haan’s December 22, 2009 posting titled:  A peek behind the scenes at Hotmail

/gfs

by Kevin Timmons,

General Manager of Datacenter Services

Global Foundation Services, Microsoft Corporation

This week our team was honored to receive the Datacentre Leaders Award 2009 for Innovation in the Mega Data Centre category for our new Dublin facility by Data Center Dynamics magazine.

 “We’re honored to receive this award, which recognizes the environmental sustainability approaches of our recently opened mega datacentre in Dublin. This facility takes a fresh approach in integrating new design solutions to achieve greater efficiencies and waste reductions, deviating from more traditional methods used in many current facilities today.” (from Declan Comerford, datacenter architect)

Back in September 2009 we opened the doors to our Dublin datacenter, the first mega facility dedicated to Microsoft in EMEA. The expansion in Dublin was a part of the company’s long term business approach to meet future demand by pre-investing in a way that would allow us to support future capacity in an incremental, cost effective manner. With the launch of our strategy for Software-plus-Services it was well understood that our datacenters would have to grow to support customer needs and demands worldwide. But we are not only committed to the idea of datacenters serving as a backbone of our services strategy, we are also committed to creating facilities that will continue deliver technical innovations and greater sustainability to the industry. 

“This data centre represents a major milestone in our ongoing investment in Europe, providing the critical infrastructure to support the delivery of our next generation of online services for both businesses and consumers.” (from John Dwyer, international datacenter manager)

The 300,000+ square foot Dublin facility takes a fresh approach to design by integrating some unconventional and localized design solutions to support greater efficiency, deviating from more traditional methods used in many current facilities today.  Some of these include:

1.      The use of free air-cooling makes our Dublin facility the first large scale implementation of its kind in the region. This design shows what is possible when you innovate locally and build to suit the natural environmental benefits around you. It has also enabled us to validate several new design concepts for use in future facilities.

2.      Removal of water based chillers and CRAC units has allowed us to reduce the water consumption to approx 1 percent of that used in similar sized facilities.

3.      From an operations standpoint, our ability to mix server densities within the server room PODs, has allowed us to further maximize the critical power available in the facility.

We also went to great lengths to ensure that we “future-proofed” this facility as much as practically possible. Our team of electrical and mechanical design engineers created an asset which is able to accommodate a wide range of applications and power densities. This design approach ensures that the Dublin facility will continue to be a core piece of Microsoft’s computing infrastructure for many years to come.

Once again, we are deeply appreciative to have received this recognition for the efforts of a lot of team members who build and manage Microsoft’s production datacenters. 

If you would like to learn more about our datacenter best practices, I invite you to explore the other blogs on this site by my fellow colleagues and to download our white papers, news, and information available on our external web site at www.globalfoundationservices.com

/kt

By Dileep Bhandarkar, Ph. D.

Rightsizing can clearly reduce the purchase price and the power consumption of a server. But the benefits go beyond the savings in capital expenditure. The lower power consumption has a big impact on the Total Cost of Ownership as shown in the Figure by reducing energy consumption and also reducing its share of the datacenter’s power allocation shown in blue.

Figure: Three-year total cost of ownership of a basic 1U server

It also surprises me that so many IT groups base their purchasing decisions on published benchmark data about processors, even though that data is often generated using system configurations that are completely unrealistic when compared to real-world environments. Most folks sit up and take note when I display the facts about these topics, because the subject is important.

Rightsizing can clearly reduce the purchase price and the power consumption of a server. But the benefits go beyond the savings in capital expenditure. The lower power consumption has a big impact on the Total Cost of Ownership as shown in the Figure.

IT departments are strapped for resources these days, and server rightsizing is something every team can do to stretch their budgets. The point of my presentations and the white paper our team is publishing today is two-fold:

1.     To quantify some of the opportunities and potential pitfalls as you look for savings, and

2.     To present best practices from our experiences at Microsoft, where the group I lead manages server purchases for the large production data centers behind Microsoft’s wide array of online, live and cloud services.

In a nutshell, the paper describes how we perform detailed analysis of our internal workloads and then select balanced servers that are optimally sized for our production scenarios. It is my hope that IT teams in other companies can use the information in our paper to justify devoting resources to characterizing internal workloads, because that is the basis of an effective rightsizing strategy.

Before I came to Microsoft to manage server definition and purchases I worked on the other side of the fence. For 17 years I focused on processor architecture and performance at Digital Equipment Corporation, and then worked for 12 years at Intel, focusing on performance, architecture, and strategic planning. It’s interesting how now that I’m a hardware customer, the word “performance” encompasses cost effectiveness almost as much as it does throughput and response time. As my colleague Kushagra Vaid and I point out in our paper, when you look up performance in the dictionary it is defined as “how well something performs the functions for which it’s intended”.  

Today’s IT departments need to address the cost and power aspects of server performance because saving money and energy efficiency are such important goals for companies these days. We hope this paper, and the data and experiences it shares will help you in this regard.

\db

The Inflection Point for Efficiency in the Data Center

By Christian Belady,
Principal Infrastructure Architect
Global Foundation Services, Microsoft Corporation

The Gartner graph (included in Part 1 of this blog) also made me think about the fact-- that had the same survey been asked 5 years ago-- it would have shown100 percent are either looking at internal metrics and/or not considering anything when it comes to energy or efficiency.  In my opinion, it wasn’t until 2006 that the industry really did go through a paradigm shift.  While there were a few of us who had been pushing efficient computing approaches for over a decade), we had limited success in moving the industry until 2006.  What happened in 2006?  I think the notion of establishing an industry efficiency metric and data center metrics were born. I thought it would be interesting to recap those milestones based on my perspective:

January 31 2006: Conference on Enterprise Servers and Data Centers: Opportunities for Energy Savings

This conference really brought Efficiency on the main stage of our industry. Andrew Fanara’s (EPA) presence was the key to waking up the industry and his goal was to drive metrics for delivering efficiency into computing. This (to me) was a pivotal day since it marked the first time we collectively got together as an industry to address efficiency.  It was no longer a voice of a few, thanks to Andrew. 

March 27, 2006: Energy Efficiency Server Benchmark Technical Workshop                                                   

This event was a follow-up meeting to the EPA conference in January. I led a work group on the Applicability of Benchmarks to the real world” with members from HP, Intel, AMD, LBNL, Cisco, Dell, Sun, Rumsey Engineers and others.  The group concluded that a benchmark should be developed with power on the y-axis and workload on the x-axis.  This recommendation is what drove the publication of the paper later that year that Jon Koomey spearheaded (with many of us in the industry co-authoring) titled “Server Energy Measurement Protocol”.  This paper was the foundation of what SPECpower is today.

April 23-26, 2006:  High-Density Computing: Trends, Challenges, Benefits, Costs, and Solutions                              

This symposium was the Uptime Institute’s first and it focused on Density trends.  However, it was this conference where I first presented an Efficiency Metric called PUE which seemed to capture the attention of many of the attendees.  As a result, I published a paper on PUE with my good friend Chris Malone later in the year at the Digital Power Forum.  At this same conference, AMD’s Larry Vertal and Bruce Shaw sat down with Paul Perez (my former VP) and I to discuss the idea of starting a consortium called the Green Grid. Ten months later the Green Grid was officially announced with one of its first whitepapers evangelizing metrics and in particular PUE.

These three events to me really started it all.  If you look at the Gartner graph, you can argue that all of the metrics were seeded in these three events and are really making an impact today.

So our industry woke up in 2006 and while the Gartner graph does show we have work ahead of us, I do think we can say that in less than four years the industry has made great strides (and perhaps I shouldn’t complain so much!).

/cb

For more information on Microsoft's datacenter best practices visit our external web site at www.globalfoundationservices.com and check out Rob Bernard's feature “Using the Power of IT to Drive Environmental Sustainability” that highlights the effort Microsoft is making in environmental sustainability, as well as how that directly relates to what world leaders are currently discussing in Copenhagen for the United Nations Climate Change Conference 2009. 

By Christian Belady,

Principal Infrastructure Architect

Global Foundation Services, Microsoft Corporation

I read a lot about what others are doing in the industry, and a recent post by Rich Miller of Data Center Knowledge caught my attention. It was titled “Gartner: Energy Woes to Worsen in 2010” and he posted a Graph from Gartner showing that only about 50 percent of the IT professionals are using some form of energy metrics in the data center.

At first I was not surprised, because as many of you heard me complain in the past, “this industry is moving too slow when it comes to efficiency”. Clearly, the adoption of efficiency metrics is quite low according to the graph.  Less than 18 percent of those surveyed are using PUE/DCiE, less than 3 percent are using SPEC Power and perhaps 7 percent or so are using Energy Star.  A little more than 20 percent are using other internal metrics…and I couldn’t help but wonder what they are.  Additionally, why are the remaining 50 percent not measuring anything, particularly in a time when sustainability seems to be top of mind for everyone.  Why do these folks seem to not care?  

Perhaps the answer lies in the fact that maybe the respondents should have been weighted by their relative data center sizes.  I would argue that any company – where a bulk of their costs are driven by their IT operations (which by the way are likely to be huge operations such as Microsoft, Yahoo, Ebay, Amazon, Google, etc.) – are further to the left on the graph because it has such a large impact on profitability. Similarly, those who have either small operations or their IT costs are low relative to other costs are further to the right. So in essence, we have a polarization of the data center industry: the Leftists and the Rightists. 

The leftists are made up of large internet businesses that live and die with online commerce and consumer services. These guys are obsessed with driving cost out of their operations, are willing to break the rules and standards, believe they can change trends, quickly adopt new ideas, and take risks to substantially change their TCO models for the better.  Because of their size, they can also hire specialists to engineer and optimize their data centers to levels never seen before. 

Conversely, the rightists are made up of the more traditional enterprises where the IT department is just a very small part of their cost structure.  In these organizations, the IT department is just expected to be there.  It’s a cost that people accept because it is small and as long as there are no issues everyone is happy. The motivation for the operation’s folks is to simply make sure nothing goes wrong.  As a result, there are no incentives to improve efficiency (especially if there is a perceived risk) and their priority is uptime. As a result, there is no reason to go to outside air economization, there is no reason to add new processes to monitor and measure efficiency, and there is no reason to look at new data center architectures that could risk downtime.  This all makes sense if you look at the following graphs in which I compare two hypothetical companies, one with most of their costs in IT Operations (Leftists) and one with only 10 percent revenue dollars are eaten up by IT operations costs (Rightists):

Click image to view larger 

This shows me why it may be that companies are still not motivated to focus on efficiency.  In the case of the Leftist company - spending resources for a 20 percent IT Operations improvement in efficiency would return an improvement of 160 percent in profitability.  However, that same improvement in the IT Operations for the Rightist company only yields an improvement of 20 percent in profitability.  In their case, it would make sense to focus on improving their other costs.   So the reality is we can’t expect everyone to think efficiency is so important! Ultimately it will boil down to how sensitive a business is to the costs of its IT operations.  So in the short term, there appears to be this polarization that has occurred. However, to Gartner’s point, this could all change if energy costs increase substantially or carbon taxes emerge.  In addition, what would happen if the big online providers (Leftists) hyper-optimized their operations and can offer cloud services at a fraction of the cost of running IT Operations that the Rightists can do themselves?  Would this make all of those who did not focus on efficiency move to the cloud to survive? 

Gartner’s survey really has made me think about this.  In fact, I hope that the next time Gartner runs this survey they also track the following:

1)      What percentage of the costs are in IT Operations?

2)      How would the business be impacted if IT costs would double or even triple?

Tune in tomorrow for my thoughts on this issue and the “inflection point for efficiency”.

\cb

A few weeks back we had the privilege of hosting a data centre visit and tour for the residential neighbours of the Dublin data centre. Given the long projected lifespan of our new facility, we’re going to be neighbours for a long time, and so we felt it was important for us to get reacquainted now that the major construction is completed. We were keen to demystify the very large complex that had been emerging right in their back yard over the past year. It was very pleasant to share some pizza, provide a little background presentation, and then show this very important audience the fruits of our innovation.   

When preparing to present a mega data centre to folks who are not generally familiar with data centres, and who in some cases are very new to computers and the Internet, you have to take a very different approach, and I have to admit to finding it an enjoyable challenge to present Microsoft’s Software plus Services strategy, the emergence of the Internet, and this particular data centre in an informative and entertaining manner. The audience ranged from people in their early 20’s who are just entering the workforce to some who have been retired for many years. A couple had engineering backgrounds and wanted to delve into the construction, mechanical and electrical systems, while others were far more interested in the creature comforts we have in place for our workforce and the impact that our arrival would have on the neighbourhood from an employment, environmental, and security perspective.  

It was fascinating to hear their questions and to listen to their feedback as we toured them through the facility showing everything from the air handling units on the roof, the server pods providing hot aisle containment in the colos, and the extensive electrical distribution systems. I never imagined how hard it would be to stay away from the many industry terms and acronyms that we treat as part of our everyday language today, and on a couple of occasions it was only those quizzical looks that alerted us to the fact that we had slipped back into Microspeak again.  

Based on their comments and questions during the visit, it was clear that the community very interested in learning about our approaches to protecting the environment, to guard customer information, and to be an asset to the neighbourhood, the county and Ireland. They asked great questions about how clean and efficient we are, how we manage physical and logical security, what financial investments we were making, and what the impact on local employment would be. They were quite vocal in their enthusiasm for the design that takes advantage of the Irish weather to dramatically improve our power efficiency, reduce our carbon emissions, and to almost completely overcome the need to consume water.  Overall, the visit was a great success, and I believe we were able to provide them with a sense of why we are here, and what makes this new facility such a unique asset to Microsoft and Ireland.  

Containers Increase IT Efficiency of Microsoft’s Cloud Computing Infrastructure  

Microsoft’s cloud computing infrastructure takes another big step forward this week with the grand opening of our Chicago data center. At more than 700,000 square feet, this facility significantly expands our ability to meet the demand generated from our Live, Online, and Cloud Computing services offerings for our customers. Combined with the grand opening of our Dublin, Ireland, data center last week to expand our capacity and network throughout EMEA, the opening in Chicago demonstrates how Microsoft is expanding capacity around the world to support its online businesses and customers.

Investing in turbulent economic times is always a tough choice – one that Microsoft is clearly making by investing in technology and innovation in a thoughtful and measured manner with an eye towards long-term growth for our customers, our shareholders, and our business. Microsoft’s data centers represent the infrastructure foundation of the company’s cloud services offerings and demonstrate how Microsoft is positioning itself to compete and succeed with an approach we call Software plus Services. Microsoft’s Software plus Services strategy is designed to create computing experiences that offer both the best of client technology and the best of the Web to connect people, data, devices, and applications.

One of many reasons we decided to hold a grand opening of the Chicago facility is to share our best practices. While in one sense our best practices are competitive advantages for Microsoft, we hope they will also help others in the industry make the cloud a safer and more reliable place that companies can trust for their operations.  Very few companies can make the infrastructure investment that Microsoft has, so we think it is important to share what we’ve learned with the industry.  

The first phase of the Chicago data center represents 30 megawatts of critical power. An additional 30 megawatts is pre-positioned for future growth. This incremental approach means customers today will enjoy top-notch performance and availability while we control costs for Microsoft and its shareholders.

One aspect of this data center that I’d like to talk about in some detail here involves our continued focus on environmental best practices.

Through the use of pre-manufactured, standard shipping containers, each of which house approximately 1,800 to 2,500 servers (as we’ve noted in previous blogs), we are able to realize greater conservation of energy and deliver new advancements in power efficiency. In addition, the isolated nature of containers enables Microsoft and its vendors to research new approaches around power and cooling alternatives to reduce energy consumption even more in the future.

Inside view of a container in the Chicago data center. 

The Chicago facility is one of the largest data centers in the world to use containers. The entire first floor is devoted to parking stalls for containers, which will eventually house more than two-thirds of all the servers in the data center. The containers plug into standard interfaces called “CBlox” that we have developed with our partners. This interface provides a kind of “plug and play” for data center containers (for those of you who have been around long enough to remember when Microsoft developed “Plug and Play” for Windows).

See the video of containers being installed in the new Chicago facility.

Containers provide further environmental benefits in that they don’t require additional packaging materials or external form factors for the thousands of servers that they house. They also require less cabling and other equipment that all add up to unnecessary waste in traditional data center designs. Containers are also a very efficient way to quickly deploy capacity.

Another best practice we are using in Chicago is water-side economization, which enables us to cool the facility without requiring the high levels of electricity typically needed to power large chillers.  Environmental sustainability is in Microsoft’s DNA and the Chicago data center serves as a great example.

I’d also like to talk about the incredible quality and amount of work that went into this data center, as well as the investments Microsoft has made in the local community throughout this project. Building this state-of-the-art facility generated approximately 3,000 construction-related jobs with a peak workforce of around 1,100 workers. More than 1.5 million man-hours of labor went into the project, and the total investment in the facility will top $500 million over time.

This week’s grand opening provides us the opportunity to thank those who helped turn our vision for the Chicago data center into reality. The facility actually began operations on July 20, delivering online services to our customers. It is gratifying to reflect on this achievement as we plan our next advances to move data center sustainability and efficiency forward for Microsoft, our customers, and the industry.

Arne

Arne Josefsberg,

General Manager of Infrastructure Services

Global Foundation Services

Microsoft

Environmental advances and expanded international computing delivered

This is a big week for Microsoft’s online, live, and cloud services as we celebrate the grand opening of our new data center in Dublin, Ireland. The Dublin facility delivers two key advances for Microsoft’s Software plus Services initiatives. One is expanded support for all our customers in the Europe, Middle East, and Africa region, thanks to Microsoft’s first mega data center built outside of the U.S.  The other is dramatically improved environmental sustainability, resulting from innovative technology that takes advantage of the naturally cool climate in Ireland.

Regarding expanded international support, the new data center is already playing an important role in helping meet growing customer demand for Microsoft services in Europe and beyond. The facility, which began operations on July 1, 2009, currently covers 303,000 square feet, with 5.4 mega watts of critical power available to deliver services to consumers and business customers. Over time, the data center can expand to a total of 22.2 mega watts of critical power to support our growing cloud services.  Combined with our other new mega data center in Chicago, Illinois—which we’ll celebrate the grand opening of next week on September 30—Microsoft is taking significant steps forward in our cloud computing capabilities.

It is important to note that at both facilities we will ramp up capacity incrementally to meet increases in customer demand.  We are making thoughtful, measured investments in order to contain costs at the same time that we build out our infrastructure to deliver a secure cloud foundation, with robust performance and availability for services around the world.

Regarding the environment and Green IT factors, Microsoft is leveraging a combination of natural environmental factors to dramatically improve the environmental sustainability of the Dublin data center as compared to traditionally-built data centers. The average temperature range year round in Dublin is between 23 to 80 degrees F (-5 to 27 degrees C) and that, combined with the use of air-side economization, results in “free cooling” (chiller-free) operations 100% year round under normal operating conditions. This in turn significantly reduces water consumption and the use of chemicals required to treat cooling towers, which are common throughout the data center industry but not required in our Dublin facility.

Air-handling units on the roof of the data center (see photos above and below) draw outside air down into the facility to cool the server rooms, and then return hot air back out to the roof. Traditional data centers, on the other hand, cool server rooms with chillers, which consume a great deal of power and water. No chillers are used in the Dublin data center. The outside air that cools the facility is usually lower than the 95 degree F limit for our server rooms. If it ever exceeds that temperature, or in the extremely rare event of external air quality issues such as a nearby fire, Direct eXpansion (DX) cooling will be used. DX is a simpler means of mechanical cooling that is normally used for residential, automotive, or light commercial applications.

As a result of these measures the Dublin facility will use less than 1% of the water that traditional data center facilities typically use on an annual basis, and will improve Power Usage Effectiveness (PUE) by approximately 50%.

These improvements are part of our continued corporate commitment to environmental sustainability. Microsoft is a proud participant and endorser of the European Union (EU) Code of Conduct for Data Centres, a voluntary program that encourages organizations responsible for the operations of data centers to utilize technologies, systems, and processes that maximize the efficient use of electricity. We are one of the largest online service providers to sign the EU’s Code of Conduct for Data Centres. Our participation includes a commitment to continuing to develop and employ innovative systems and processes that deliver computing scale to meet our evolving Software plus Services business initiatives with the lowest possible consumption of power and natural resources. We also work actively with The Green Grid and Climate Savers Computing industry consortiums and the U.S. Environmental Protection Agency to advance sustainability and reduce the carbon footprint of data centers globally.

On September 16 we were honored to be recognized as a Best Practice in the European Commission's Sustainable Energy Europe Campaign for innovative achievements in design and operational strategies that are driving greater sustainability and efficiency at the Dublin facility.

While our best practices are competitive advantages for Microsoft, we hope they will also help others in the industry make the cloud a safer and more reliable place that companies can trust for their operations.  Very few companies can make the infrastructure investment that Microsoft has, so we think it is important to share our best practices with the industry. This is one of many reasons we decided to hold a grand opening of the facility today.

I’d like to close by noting how this week’s grand opening helps celebrate and build upon Microsoft’s long-term investment in operations in the Republic of Ireland. Microsoft Ireland is now nearing the 25-year anniversary mark, and I believe the building of the Dublin data center will long be remembered as a key milestone of our relationship with this community. The $500 million facility is one of the largest construction projects in Ireland over last 12 months and has generated approximately 1 million man-hours of work with a peak workforce of around 2,100 workers. The data center will also provide approximately 35-50 jobs in the Dublin area for years to come.

It is exciting to unveil this marvellous facility and I’d like to extend my thanks to all the people inside and outside Microsoft who helped make it happen.

Arne

Arne Josefsberg,

General Manager of Infrastructure Services

Global Foundation Services

I recently participated in a Podcast for InsideHPC.com with a discussion on “the differences on going green in general IT and going green in HPC” which John West put so eloquently  (go to Episode 2: IT, HPC, and where the twain shall meet at http://tinyurl.com/m7ku4m).  I found this to be an interesting discussion for several reasons.

First, as many of you know, I have been saying for over a decade now that efficient computing will be a differentiator. I believe this is true now more than ever before.  In this economy, capital resources are constrained to levels organizations have never seen before.  As I stated in the Podcast: that is where the opportunity lies.  I incorrectly quoted my favorite inventor Thomas Edison as saying “Innovation only occurs with constraints.”  What he really said was “Discontent is the first necessity of progress” and it was actually Mike Heffernan (High Tech Visionary) that stated “The absence of constraints is the death of creativity.”  So what I said in the Podcast captures the intent for both of these visionaries.  The constraints the economy has place on IT budgets has driven innovation through necessity. For enterprises and cloud services businesses such as ours, we have to drive costs down since margins are tighter than ever before. Similarly, HPC is facing less funding but its computation needs are exploding -- resulting in focus on efficiency and innovation to meet needs with smaller budgets.  It’s really quite exciting to see businesses being more proactive in this climate.

Second, I have spoken about the fact that at Microsoft, as with our competitors, we are driving towards hyper-efficiency and moving away from acquiring general purpose servers. Clearly our business focus over the past couple of years for our business has been to reduce cost by right sizing to our needs.  Anything that is not needed is stripped to improve efficiency and reduce cost.  I did not think of this before, but Steve Cumings made the statement that HPC is requiring the same. 

Finally, Pat Tiernen said something which I believe is very important.  “If it will improve your business anyway, why not do it.”  He is absolutely right. It’s astounding to me that organizations aren’t naturally trying to become more efficient just for the simple reason that it will drive costs down.  Why is it that utility companies need to be giving incentives for people to save money?  Look at PG&E, I simply applaud Mark Bramfitt for leading the charge in this space.  He gives you money to save money.  What a concept. J 

So in the end, ultimately, the economic constraints put on us as an industry will finally provide the motivations to drive towards efficiency.  It’s not about differentiation any more, it’s about survival.  There is no stronger incentive than that.

cb

Christian Belady, Principal Infrastructure Architect

Global Foundation Services

I am really excited that our team is now reaching another key milestone in data center innovations. July marks the launch of our two newest mega data centers in Chicago and Dublin. Our Dublin facility will go live on July 1, followed by our Chicago facility on July 20 to support our growing Online, Live, and Cloud services.

Together these Generation 3 facilities demonstrate Microsoft’s continuing commitment to improving data center efficiency with a focus on environmental sustainability.

The Dublin, Ireland, data center is our first mega data center built outside of the United States. This building covers 303,000 square feet, with 5.4 mega watts of critical power available now. Over time, the data center can expand to a total of 22.2 mega watts of critical power, growing with our business and customer demand. The facility makes extensive use of outside air economization to cool the facility year round, resulting in greater power efficiency with a resultant reduction in carbon footprint.

                                           Dublin Data Center

The Chicago, Illinois facility covers over 700,000 square feet—approximately the size of 16 football fields—with critical power of 60 megawatts. Phase 1 represents 30 mega watts of critical power and the rest is pre-positioned for future growth. Two-thirds of the Chicago data center is optimized for housing containerized servers. Containers conserve energy and will help us realize new advancements in power efficiency with a PUE yearly average calculated at 1.22.  These prepackaged units (with up to 1,800 to 2,500 servers each) can be wheeled into the facility and made operational within hours, so they represent important advances in the ability to quickly and efficiently provision capacity. The density inside the containers can exceed 10 times that of traditional data centers.

It is truly exciting to bring these two data centers online. We take great pride in the innovations they deliver to move the data center industry forward and to extend Microsoft’s online services to customers globally.  As the company’s Software-plus-Services strategy progresses, these data centers will play a key supporting role.

I’d like to extend my congratulations and thanks to everyone at Microsoft who was involved in creating these state-of-the-art facilities and to all the local and regional trades people and organizations that helped make them possible. For more information on our cloud infrastructure strategy and services please visit our web site.

Arne

Arne Josefsberg,

General Manager of Infrastructure Services

Global Foundation Services

Building an organization around exceptional leaders with the deepest industry expertise is core to how we evolve our Global Foundation Services organization. One such leader is Kevin Timmons, who joined Microsoft Global Foundation Services (GFS) today to head up our Data Center Services organization. Kevin brings a wealth of knowledge and passion in this space, most recently serving as vice president of Operations at Yahoo!, where he led the build-out of their data centers and infrastructure. Before that he was a director of Operations at GeoCities, and prior to that he served as a senior software engineer at Marconi Dynamics.

Kevin is known as a hands-on leader with a great grasp on the issues in his field and a keen interest in increasing energy efficiency. One of the key ways he has approached that challenge was by closely measuring efficiency at each data center and using PUE (Power Usage Effectiveness) as a key metric—a strategy that helped build more efficient data centers. Kevin also brings valuable experience and know-how in the field of data center site selection. Anyone familiar with our areas of focus in Microsoft data centers, especially around environmental sustainability, will recognize the great fit Kevin’s experiences bring to the team. In sitting down with him and exploring these areas in depth, I’ve become increasingly excited about the industry experience Kevin brings to our team.

In addition to bringing Kevin on board, we’ve recently restructured our Infrastructure Services team within GFS. In mid May we aligned the organization around five teams: Shared Infrastructure, Programmable Infrastructure, Platform Hardware and Standards, Global Network Services, and the Data Center Services team that Kevin now heads up. These changes will help us align our teams with Microsoft’s evolving cloud computing business and position our infrastructure for the upcoming year and beyond. We look forward to sharing more about these teams in upcoming postings.

The release last week of our white paper on Securing Microsoft’s Cloud Infrastructure generated a lot of discussion in the industry, which was our intent. We wrote the paper in part to communicate our practices to customers concerned about security in the cloud environment and to generate a healthy dialogue within the industry in order to share best practices for creating more secure cloud-based services.

Pete Boden, general manager of our Online Services Security team, posted a blog today providing more detail in response to questions we've gotten since we released the paper. Give it a read to learn more about Microsoft's history in online services and security, and how that history led to our Information Security Program that has been independently certified by British Standards Institute (BSI) Management Systems America as being compliant with ISO/IEC 27001:2005.

Companies considering moving their IT operations to the cloud are understandably concerned about security, privacy, reliability, and operational controls. To address these concerns and help share best practices with the rest of the industry, the Online Services Security and Compliance (OSSC) team within the Global Foundation Services group that supports Microsoft’s cloud infrastructure has published a white paper titled: Securing Microsoft’s Cloud Infrastructure

Our Online Services Security team is led by Charlie McNerney, GM, Business & Risk Management, who has posted a blog today describing how our coordinated and strategic application of people, processes, technologies, and experience with consumer and enterprise security has resulted in continuous improvements to the security practices and policies of the Microsoft cloud infrastructure.

And you can find more information on our strategic vision for Microsoft’s infrastructure at our Global Foundation Services web site.

Part # 2

(A couple of years ago, when our Data Center Services’ Research & Engineering team within Microsoft’s  Global Foundation Services (GFS) group kicked off the Generation 4.0 Data Center design project,  we began with the question of:  What are the primary business challenges facing data center deployments today? )  Here’s the rest of the story….

Question Everything

Often the application of technology involves as much innovation as the technology itself.  Rather than reinvent the wheel we looked at our industry’s journey thus far.  We started by questioning everything up to and including the roof and the very definition of a data center. With a lot of prior art in modularization, some of which has already been applied to the IT industry, we saw a good fit. The military has been deploying portable ground stations with IT servers and communications equipment for decades. And for some time now telecom companies have deployed pre-manufactured buildings which are then assembled on site as central offices.

As part of our design project we met to discuss the modular solution space with a group of folks from our Infrastructure Services team within GFS , including members of our hardware, data center operations, development, engineering, risk management, and security teams, as well as our internal product groups. (The original Gen 4.0 team from the two-day session is 100 percent intact and still working at Microsoft, by the way.) We knew we could modularize the server room, which we now call Server PACs. The challenge became how to modularize the entire facility. So we created other PACs: Generator PACS, Medium Voltage Switchgear PACs, UPS PACs, etc. Next, we developed the system electrical one-line diagrams and mechanical schematics for our four data center classes.

Then we had to take these single lines and schematics and break them into logical modules for the components to reside in. This may seem easy but represents a shift in thinking from a building where, for instance, we would have a UPS room and associated equipment and switchgear manufactured by multiple vendors and put it physically in sometimes separate modules. The challenge became how to shift from a traditional construction mindset to the new, modularized manufacturing mindset. Maintainability is a large part of reliability in a facility, and became a key differentiator between the four classes. Our A Class infrastructure, which is not concurrently maintainable and is on basically street power and unconditioned air, will require scheduled downtime for maintenance. The cost, efficiency, and time-to-market targets for A Class are very aggressive and a fraction of what the industry has come to see as normal today.  We realized that standardization and reuse of components from one class to the next was a key to improving cost and efficiency.  Our premise was that the same kit of parts (or modules) should be usable from class to class. These modules (in this new mindset) can be added to other modules to transition within the data center from one class to the next.

 Standardize to Differentiate

With the standardization of a kit of modular parts it is possible to supply different types of facilities—large-scale, mega-data centers and edge or mini-data centers—from the same supply chain.  A pre-manufactured set of solutions can thus reduce costs through economy of scale. Traditionally, because of network and site service costs it has been more cost effective to deploy 40 megawatts of capacity in a single location than to build 40 individual 1-megawatt facilities. In a pre-manufactured model that isn’t necessarily the case.  The use of a production-line approach, while at the same time simplifying interconnections through the right modularization of the components, can drive costs down for mini- as well as mega-data center facilities. Of course the fixed costs of site development must be considered, but we are finding that the production-line approach is significantly narrowing the difference in cost per megawatt for mega- and mini-data centers.

We realize that innovation will come from many sources, and so we are partnering with technology leaders in the vendor community, as well as our customers, to help drive and deliver the benefits of this new approach. We believe it is crucially important to develop this technology now due to the many constraints on our environment and economy globally. However, we do not have unlimited resources to drive this alone. Through industry partnership and collaboration we will be able to accelerate the adoption of modular data centers and deliver the benefits of technology, software, and services to more people.

Imagine the possibilities of bringing affordable computing to corners of the world that have no power or IT grid to speak of today. This could be done using modular power plants such as fuel cells and modular data centers in partnership with wireless technology where there is a lack of existing infrastructure (for example). Applications could become virtual in the local modular data center, or the cloud, and the thousands of services and applications it holds could be provided via low-cost devices. Great opportunities like being able to provide a $100 or a lot less laptop (or new access devices not yet invented) per child could then be achieved sooner.  A modular data center could quickly be deployed to support millions of virtual applications and at the same time access the power of the Internet and cloud services for a large population that does not have that opportunity today. Personally, I am very excited about these possibilities and believe I am fortunate to be working on this technology at this time.

We look forward to continued collaboration with our industry participants.

You can read part 1 of this 2 part series in my earlier blog on April 29, 2009

You can watch the Generation 4 video at this web site link

/dc

Daniel Costello, director of Data Center Services 

Global Foundation Services

Daniel Costello is the director for Data Center Services at Microsoft, responsible for data center research and engineering, standards and technologies, data center technology roadmap, Generation 4 data center engineering, data center automation and integration with IT hardware, operating systems and applications.  Daniel works closely with Microsoft Research on proof of concepts in support of the data center of the future and manages a team of facility engineers and service architects.