Windows Server Blog

Windows Server 2012, PowerShell 3.0 and DevOps, Part 1…

2012-05-29T16:55:00Z

In the first of a two part series, I provide some background information about PowerShell and DevOps. In the second blog, I’ll provide you a bunch of specifics. PowerShell 3.0, like Windows Server 2012, as a ton of new features and enhancements so I’ll only scratch the surface.

--Jeffrey

The first time I heard DevOps was a podcast describing the 2009 Velocity conference. While most of the industry was struggling to deploy releases a few times a year, John Allspaw and Paul Hammond rocked the house with the talk “10 Deploys Per Day: Dev And Ops Cooperation at Flickr”. They made the case for delivering business results through changes in culture and tools, and gave birth to a new term: DevOps. The problem is that developers think they are responsible for delivering features and operators are responsible for keeping the site running. The gap between developers and operators leads to finger-pointing when things go wrong. Successful business requires an IT culture of joint accountability and mutual respect: developers thinking about the needs and concerns of operators and operators thinking about the needs and concerns of developers.

Their talk described how businesses required rapid change but that change is the root cause of most site-down events. Shunning the traditional “avoid change” approach, they advocated minimizing risk by making change safe through automation. This is the job of DevOps – safe change. This was the Taguchi quality approach applied to IT operations. Taguchi observed that the root cause of poor quality was variation. The solution was to first figure out how to do something repeatably. Once you could do that, then you can make small modifications in the process to see whether they make things better or worse. Back out the changes that make things worse. Keep doing the things that make things better. The key is repeatability. Repeatability allows experimentation which drives improvement. We get repeatability in IT operations through automation.

We started PowerShell by publishing the Monad Manifesto which articulated the problems we saw, our approach to solving them and the components we would deliver. We envisioned a distributed automation engine with a scripting language which would be used by beginner operators and sophisticated developers. PowerShell’s design was driven by the same thinking and values that drove the birth of DevOps:

Focus on the business
Make change safe through automation
Bridge the gap between developers and operators

Focus on the business
PowerShell has always focused on people using computers in a business context. PowerShell needed to be consistent, safe, and productive. Much has been made of the similarities between PowerShell and UNIX but in this regard, our ties are much closer to VMS/DCL and AS400/CL.

Consistent: Operators and developers don’t have a lot of time to learn new things. A consistent experience lets them to invest once in a set of skills and then use those skills over and over again. PowerShell uses a single common parser for all commands and performs common parameter validation delivering absolute consistency in command line syntax. PowerShell cmdlets are designed in a way that ubiquitous parameters can provide consistent functions to all commands (e.g. –ErrorAction, –ErrorVariable, –OutputVariable, etc)

Safe: An Operator once told me that occasionally he was about to do something and realized that if he got it wrong, he would be fired. In PowerShell, if you ever execute a cmdlet which has a side-effect on the system, you can always type –WhatIf to test what would happen if you go through with the operation. We also support –Confirm, -Verbose and –Debug. Despite these safeguards, things can go wrong and when they do, PowerShell spends a lot of effort to speed up the process of diagnosing and resolving the error.

Productive: Every aspect of PowerShell’s design maximizes the power of users (ergo the name). PowerShell makes it easy to perform bulk operations across a large number of machines. PowerShell also makes it easy to have productive engagements between your operators and developers because it allows them to speak a common language and to help each other with their scripts.

Make change safe through automation
There has been a lot of discussion about whether PowerShell is a .Net language, a scripting language, or an interactive shell. PowerShell is a distributed automation engine with a scripting language and interactive shell(s). Interactive shells and a scripting language are critical components but the focus has always been on automation through scripting. Automation is the process of reducing and/or eliminating operations performed by a human. A script documents what is going to happen. People can review a script and you can modify it based upon their feedback. You can test the script, observe the outcome, modify the script and if modification is good, keep it and it if is bad back it out. In other words, scripting provides the repeatability required to apply the Taguichi method to IT operations. Once you have an automated process, you can safely apply it over and over again. These processes can now be performed reliabily by lower skilled admins. These steps aren’t possible when you use traditional GUI admin tools.

Bridge the gap between developers and operators
Our goal has always been to deliver a single tool which could span the needs of operators doing ad hoc operations, simple scripting, formal scripting, advanced scripting and developers doing systems-level programming.
PowerShell spends a ton of effort trying to project the world in terms of high level task-oriented abstractions with uniform syntax and semantics. We call these cmdlets. And this is what operators want to efficiently and effectively manage systems. In order to copy a file using APIs, you would do this:

Have you ever wondered why PowerShell uses curly braces {} (and other C constructs) instead of BEGIN/END as other scripting languages do? We did that because we wanted to make it easier to adopt by developers of other C-based programming languages: C++, Objective C, Java, JavaScript, Perl, PHP, etc. We did some testing and determined that operators were able to readily adapt to this syntax. We also wanted to provide a smooth glide path between PowerShell and C# . This provides career mobility for operators who might want to transition to being a developer.

Most importantly, we wanted to develop a tool which could be used by BOTH operators and developers to bridge the gap between the groups and allow them to create common scripts, learn from each other and work together.

Windows Server 2012 and PowerShell 3.0 are excellent DevOps tools
DevOps is a new term and there is some disagreement about what it entails but at the heart it is all about making change safe through automation and bridging the gap between operators and developers. There is a lot to do in this area but Windows Server 2012 and PowerShell 3.0 make excellent progress towards accomplishing those goals. PowerShell won’t be the only tool in your DevOps toolbox but it should be in every DevOps toolbox. Download the beta today and find out for yourself.

Cheers!

Jeffrey Snover
Distinguished Engineer and Lead Architect for Windows Server

Announcing the Windows Server 2012 Community Roadshow

2012-05-24T16:24:49Z

We’ve put a lot of work into Windows Server 2012, and are happy to see the positive feedback from press and those who’ve downloaded the beta. We look forward to presenting Windows Server 2012 at our June TechEd events in Orlando and Amsterdam. In this blog, Christa Anderson, a Program Manager in our Partner and Customer Ecosystem team, talks about an opportunity for those that can’t make it to those events.

-Cheers! Jeffrey

Not everyone who wants to learn in-depth about Windows Server 2012 can make it to TechEd. Therefore, in cooperation with our co-sponsors Dell and HP, we’re sponsoring the Windows Server 2012 Community Roadshow, a complimentary series of events held around the world to present on some of the key infrastructure improvements we’ve made in Windows Server 2012, including:

Server Virtualization
Networking
Storage
Server Management and Automation

Even better, this road show will be presented by Microsoft MVPs: independent technology experts and experienced presenters. MVPs speak regularly at third-party and Microsoft industry conferences such as TechEd, so we’re thrilled to have so many MVPs presenting in this road show.

You can sign up at no cost for technical sessions in your area from the Windows Server 2012 Community Roadshow web site. Some events are already full, but you can get on the wait list by clicking the SOLD OUT link and following the instructions.

This is a great opportunity to learn about core infrastructure improvements in Windows Server 2012 from independent technical experts. Go register!

Christa Anderson

Introduction to Windows Server 2012 Dynamic Access Control

2012-05-22T19:05:00Z

We constantly strive to reduce the steps required for you to get your job done. One of the reasons Windows Server 2012 is a such great release is that we spent so much time listening to our customers and understanding their scenarios and concerns. When development teams start from a technology/feature mindset, it can be hard to work across groups because helping another team usually means that you have to give up something you wanted to do. We were able to achieve a very high level of technology integration and cross-group cooperation because we all shared a common understanding of our customers and their scenarios. Teams were eager to help each other succeed in delivering those scenarios. When you have lots of teams working together towards a common goal, you can really change the game and tackle some really hard problems. Today’s blog is a good illustration of that.

Anyone that has been involved in securing data or accessing data security knows that the traditional security models and mechanisms are not always flexible enough to address today’s concerns and scenarios. Whether it's compliance requirements, increased business impact of disclosed data, or management of the sheer scale of data – it is clear that the capabilities provided by the current access control mechanism can be improved so that it is easier for administrators and users to address these challenges. A number of teams worked together to deliver Windows Server 2012’s Dynamic Access Control. I think you’ll find that it, like so many other things in Windows Server 2012, is just what you were asking for.

If you haven’t downloaded the beta yet, take some time to read this blog and watch some of the videos it points to and then schedule some time on your calendar to download the beta and try it out.

Nir Ben-Zvi, a Program Manager on the File Server team, wrote this blog.

--Cheers Jeffrey

Hello, my name is Nir Ben-Zvi and I work in the Windows Server team. I’m very excited to introduce to you the new Dynamic Access Control feature set.

I’ll start with a brief introduction and insight into the planning process, discuss the new Central Access Policy model and describe the end-to-end File Server solution that we built into Windows Server 2012. I will also touch on how we enable an incremental deployment model so that you do not need to move your entire environment to Windows Server 2012 in order to use the feature set. I will touch on work with partners in this area, too.

You can find a Dynamic Access Control overview demo here.

Introduction

In today’s complex IT environments data is being created on distributed systems at a staggering rate and accessed through a plethora of devices. Compliance with regulatory standards and the need to secure leakage of business critical and personal data present major challenges for businesses and corporate IT. The key pillars for data compliance and leakage prevention are controlling who has access to information and having the ability to report who actually accessed specific information.

Not surprisingly, this was the exact situation that we observed when we started planning for Windows Server 2012 a few years ago. A few of the points that we repeatedly heard from customers during the planning period included:

Centrally manage access to information based on business and compliance needs
Access to information needs to be audited for compliance and analysis purpose
Sensitive information should be protected wherever it goes
Content owners should be responsible for their information - IT admins are not librarians
Maintaining information worker productivity is key

We then looked at the different personas within an organization and how they participate in meeting the regulatory and business requirements for data compliance, in order to provide the right set of technologies and solutions that help address the data compliance challenge.

The range of personas involved starts with the CSO/CIO office that identifies the business and regulatory compliance needs. It continues with the IT administrators that manage the systems and the business owners that control the actual information. Last, the organization would like to keep the impact on the information worker to a minimum (ideally with no impact at all).

To help organizations reach their data compliance, we eventually focused on the following areas:

Identify the information that needs to be managed to meet business and compliance requirements
Apply appropriate access policies to information
Audit access to information
Encrypt information

These focus areas were then translated to a set of Windows capabilities that enable data compliance in partner and Windows-based solutions.

Add the ability to configure Central Access and Audit Policies in Active Directory. These policies are based on conditional expressions that take into account the following so that organizations can translate business requirements to efficient policy enforcement and considerably reduce the number of security groups needed for access control:

Who the user is
What device they are using, and
What data is being accessed

Integrate claims into Windows authentication (Kerberos) so that users and devices can be described not only by the security groups they belong to, but also by claims such as: “User is from the Finance department” and “User’s security clearance is High”
Enhance the File Classification Infrastructure to allow business owners and users to identify (tag) their data so that IT administrators are able to target policies based on this tagging. This ability works in parallel with the ability of the File Classification Infrastructure to automatically classify files based on content or any other characteristics
Integrate Rights Management Services to automatically protect (encrypt) sensitive information on servers so that even when the information leaves the server, it is still protected.

Central Access Policies

One can look at Central Access Policies as a safety net that an organization applies across its servers. These safety net policies enhance (but do not replace) the local access policy (e.g. Discretionary ACL) that is applied to the information. For example, if a local DACL on a file allows access to a specific user but a Central Policy restricts access to the same user, the user will not be able to get access to the file (and vice versa.)

The initiative to deploy and enforce a Central Access Policy may come for different reasons and from multiple levels of the organization:

Compliance policy: This policy relates to compliance and business requirements and is targeted at protecting the right access to information that is being managed. For example: Allow only a specific group of people access to information that falls under the “US-EU Safe Harbor” regulation.
Departmental authorization policy: Each department in an organization has some special data handling requirements that they would like to enforce. This is very common in distributed organization. For example: The finance department wants to limit all access to finance information only to the finance employees.
Need to know policy: This is a policy that ensures that access is allowed on a need-to-know basis. Examples include:

Vendors should be able to access and edit only files that pertain to the project that they are working on.
In financial institutions, information walls are important so that analysts do not access brokerage information and brokers do not access analysis information.

Central Audit Policies

Central Audit Policy is a powerful tool to help maintain the security of an enterprise. One of the key goals of security audits is regulatory compliance. Industry standards such as SOX, HIPPA, PCI, etc. require organizations to follow a strict set of rules related to information security and privacy. Security audits help establish the presence (or absence) of such policies and thereby prove compliance (or non-compliance) with these standards. Additionally, security audits help detect anomalous behavior, identify and mitigate gaps in security policy and deter irresponsible behavior by creating a trail of user activity that you can use for forensic analysis.

Windows Server 2012 enables administrators to author audit policies using expressions that take into account what information users are accessing and who the user is so that an organization can target audit to specific information wherever it resides. This opens the doors to richer, more targeted and easy-to-manage audit policies. It enables scenarios that until now were either impossible or too difficult to enable. For example you can now easily author audit policies such as the ones listed below:

Audit everyone who does not have a high security clearance and yet tries to access “high impact” information.
Audit all vendors when they try to access documents related to projects that they are not working on.

This helps regulate the volume of audit events and limit them to only the most relevant information/users so that you can monitor access to information across multiple servers without generating an unmanageable volume of audit events.

In addition, the information tagging is recorded in the audit events so that event collection mechanism can provide contextual reports such as: Who accessed all the “high impact” information in the last three months.

The File Server solution

Based on this infrastructure we built a full end-to-end Windows-based solution for Windows Server 2012 Active Directory, Windows Server 2012 File Server and Windows 8 client. This solution allows you to:

Identify data using automatic and manual classification of files.
Control access to files across file servers by applying safety net Central Access Policies. For example, you can control who can access health information within the organization.
Audit access to files on file servers by using Central Audit Policies for compliance reporting and forensic analysis. For example, you could identify who accessed highly sensitive information during the last three months.
Encrypt data by automatically applying Rights Management Services (RMS) encryption for sensitive Microsoft Office documents. For example: you could configure RMS to encrypt all documents that contain Health Insurance Portability and Accountability Act (HIPAA) information.

In order to support deployment across multiple file servers in the organization, we are also providing the Data Classification Toolkit that enables configuration and reporting across multiple servers.
The current Beta for the Data Classification Toolkit is available for download here.

The concept of incremental deployment

One of the core design principles of Dynamic Access Control is incremental deployments. You can start using the feature set as soon as possible to solve targeted business problems for information access and audit.

You can use most of the Dynamic Access Control capabilities with the Windows Server 2012 File Server and an upgraded Active Directory domain schema. Adding a minimal number of Windows Server 2012 domain controllers will enable user claims and so on. Each part of the system that you upgrade provides you with more capabilities but it is up to you to set the pace.

Partner solutions

Partner solutions and line of business applications can further use the Windows infrastructure investments for Dynamic Access Control, providing great value for organizations that use Active Directory. A few examples of partner solutions that we have already demoed at the //build/ conference last year include:

Data Leakage Prevention (DLP) integration for automatic content classification
Central audit analysis
Rights Management Services (RMS) authorization using Central Access Policies
Many others…

We plan to show many additional partner integrated solutions in the upcoming TechEd US conference (Jun. 11-14, 2012) Twitter hashtag #MSTechEd

A few additional resources that you might find useful:

TechNet manual (Beta): http://technet.microsoft.com/en-us/library/hh831717.aspx

Data Classification Toolkit (Beta): https://connect.microsoft.com/site715

Hands on lab: http://technet.microsoft.com/en-us/windowsserver/hh968267.aspx (Using Dynamic Access Control to automatically and centrally secure data)

Dynamic Access Control at MMS 2012: http://channel9.msdn.com/posts/Dynamic-Access-Control-Demo-and-Interview

Nir Ben-Zvi

Improved Server Manageability through Customer Feedback: How the Customer Experience Improvement Program makes Windows Server 2012 a better product for IT Professionals

2012-05-17T18:38:00Z

I once talked to a doctor who told me about a recent patient that had serious medical symptoms for over a year before visiting the doctor. He said that if the patient had mentioned these symptoms when they first arose, the prognosis was very good but now the patient was in trouble. That reminded me of some advice I once heard, “Never hold anything back from your doctor”. Doctors have exactly one job: to help you. They can only help you with problems that they know about so if you aren’t completely open and honest with them, you are only hurting yourself. The other thing is that by sharing your situation with a doctor, the doctor gains knowledge and skills to help other people as well. This model and thinking applies to our Customer Experience Improvement Program (CEIP) for Windows Server 2012 Beta. That is where we ask you to allow us to collect data about the health and usage of your servers. We frequently receive questions about CEIP; ‘what is CEIP?’ and ‘how is CEIP data used?’. In this post, Karen answers these questions along with the most important question ‘why should I enable CEIP?’

Karen Albrecht, a Program Manager on the Windows Server Telemetry team, authored this post.

--Cheers! Jeffrey

When we talk to the server community about the Windows Customer Experience Improvement Program (CEIP), most people say ‘Never heard of it’. Those that have heard of it sometimes don’t enable it because they ‘don’t want to share their data’. In this blog article we will explore what CEIP is and what benefits you may receive by enabling it on your deployed Servers. We will also discuss several new features in Windows Server 2012 that make it easier to enable CEIP.

Let’s start by answering the question ‘What is CEIP?’ For those who have never seen CEIP before, using Windows Server 2012 Beta you can get there through Server Manager -> Local Server -> select the Customer Experience Improvement Program link.

CEIP is the program by which we learn how you use Windows Server 2012, in order to improve the product based on your feedback. You can join the Windows Server 2012 CEIP program in several ways. First, for pre-release beta software, such as the Windows Server 2012 Beta, CEIP is enabled by default to help us improve the software before its’ final release. Alternatively, in released products such as Windows Server 2008 R2 we provide notice through the CEIP user interface (shown above) so you can elect to opt-in to the program.

We know that you need to get the most out of your servers, especially when it comes to server performance and network bandwidth. The CEIP report collection and transfer process are light weight in order to meet this need. Windows records CEIP usage information using a high-speed tracing component, Event Tracing for Windows (ETW). ETW enables Windows Server 2012 to write out CEIP usage data no noticeable impact to server performance. CEIP usage information is transferred to Microsoft in a two part process using the Consolidator and Uploader scheduled tasks. The consolidator exports CEIP data into a compressed binary format that is ready for transfer. The binary is typically less than 1 MB in size so that the transfer has minimal impact to network bandwidth. The uploader scheduled task runs every once every 24 hours and transfers the CEIP binary data to the Microsoft frontend servers using the Windows Telemetry Protocol.

Another question we are often asked is ‘What data is collected by CEIP’? The data consists of basic information about how your server is configured and used; roles installed, features installed, settings used, and information about hardware. CEIP does not intentionally collect Personally Identifiable Information (PII). So, CEIP reports do not contain your contact information, such as your name, address, or phone number. This means CEIP will not ask you to participate in surveys or to read junk e-mail and you will not be contacted in any other way. The Microsoft Customer Experience Improvement Program privacy statement discusses, in detail, the data collected by CEIP and how we use it.

Moving on to the heart of the question, ‘What do I get for sending this data to Microsoft?’, you might be surprised in the ways Windows Server uses your data to improve the product. There are many examples beyond what is listed here. However we narrowed it down to the following to give you a flavor of some of the ways CEIP data is used to improve the product.

Increased server reliability: In the Windows Server 2012 Developer Preview and Windows Server 2012 Beta pre-release versions, Reliability Analysis Component (RAC) features are enabled to determine the root cause of Windows server crashes, Windows server hangs, and application crashes. RAC combines CEIP data with Windows Error Reporting (WER) data in order to reconstruct a full view of the system state at the time of the crash or hang. By analyzing the combined data in these two programs we can identify high occurrence issues in order to triage and fix them so that you have a more reliable platform release over release. To learn more about the data collected by WER, see the Microsoft Error Reporting Privacy Statement.
Improved programmability for server administration scripts: For large scale deployments, IT administration is often done using PowerShell and WMI scripts because scripting simplifies manageability at scale. When a commandlet or WMI interface changes or is removed, it can be painful to rewrite scripts to accommodate the platform changes. In Windows Server 2012 we are using CEIP to address this by monitoring deprecated API usage so that APIs are not removed until it has minimum impact to you. As an example, in Windows Server 2012 the Win32_ServerFeature WMI interface had been considered to be deprecated and being replaced with MSFT_ServerManagerDeploymentTasks. (For those who haven’t used it, Win32_ServerFeature detects installed roles and features.)

As part of the deprecation process, we added CEIP data to record interface usage and based on the latest Windows Server 2012 Beta CEIP data, we found that 47% of customers are using Win32_ServerFeature. Using this data, we are able to identify migration off of Win32_ServerFeature so that it is not formally removed from the product until migration to MSFT_ServerManagerDeploymentTasks can be done without impact to you.
Diversity of Windows Certified hardware: One of the frequently asked questions we get is ‘What CEIP data does Microsoft share with partners?’ There are certain scenarios where a subset of CEIP data (but no PII) is shared with IxVs (independent hardware or software vendors) as part of hardware certification. An important part of the Windows server offering is supporting high quality drivers for a diversity of devices in market. The challenge is to understand what devices are most commonly used in market. CEIP data is used to model hardware profiles and map diversity of different devices in order to inform certification strategy for IxVs. Using this data, IxVs determine the breadth of drivers to certify (based on what is in market) and prioritize which devices get certified first (based on popularity).
Improved product experiences: CEIP data is used on a day-by-day basis to understand a broad range of feature configurations so that we can prioritize work according to your usage patterns. For example, in order to reduce the cost to setup new servers, CEIP records what settings you use. This allows us to refine default settings by tuning them to reflect most common usage patterns so it is faster for you to setup a new server. Another example of internal usage is in testing. In order to increase test coverage of real world test patterns, we analyze CEIP data to understand how the product is used. This ensures that both design and testing are driven with your usage patterns in mind. There are many, many more examples of how CEIP is used to drive customer feedback into the product but in the interest of time, let’s move on to how to configure CEIP.

After the release of Windows Server 2008 R2, we did an assessment of CEIP adoption and found that 5-7% of servers in market were reporting CEIP. While working with customers on CEIP adoption we found that although servers were opted-in we weren’t getting data from them. We did a root cause analysis and learned that the main reason servers weren’t reporting is because they are deployed in firewalled environments. To send CEIP data, servers need to be able to communicate over HTTPS (default port 443) and need to have proxy settings configured (if the server is in a network that uses a proxy server). In working with Technology Adoption Program (TAP) customers, we found that frequently one or more of these settings were not configured, thus preventing CEIP data from reaching Microsoft.

To make it easy to send CEIP data, Windows Server 2012 Beta ships several new features that allow you to get past the blocking issues so you can ‘set and forget’ CEIP. To participate in the CEIP program, the simplest way to deliver CEIP data to us is to use a new feature called Windows Feedback Forwarder (WFF). WFF is a service that proxies CEIP data from machines in a domain to Microsoft. WFF will proxy CEIP data Windows products including Windows 7 and Windows Server 2008 or higher. WFF will also proxy data for any Microsoft product that is enabled to ‘send customer feedback’.

The forwarder can sit within the domain or as an edge server. Machines in the domain are configured to send data to the forwarder via group policy. When an individual machine is triggered to collect data, it sends the data to the forwarder over HTTP and the forwarder relays the data to Microsoft over HTTPS.

To install Windows Feedback Forwarder

Using the User Interface (UI)

On any Windows Server 2012 machine, launch Server Manager and then launch the Add Roles and Features wizard.
In the Add Roles and Features Wizard, navigate to the Features page, select Windows Feedback Forwarder.
Specify an incoming port number (default port number is 53533). If the domain has an internet proxy, specify the proxy information. Finish the install.
In Server Manager, select ‘All Servers’ in the left hand navigation pane. In the ‘Servers’ tile, right click the server that you installed Windows Feedback Forwarder on and select ‘Windows Feedback Forwarder Configuration’. Keep the dialog open for the next step.

OR Using PowerShell

Launch PowerShell and run ‘Add-WindowsFeature WFF’
In Server Manager, select ‘All Servers’ in the left hand navigation pane. In the ‘Servers’ tile, right click the server that you installed Windows Feedback Forwarder on and select ‘Windows Feedback Forwarder Configuration’.
Select the ‘Forwarding Settings’ tab and specify an incoming port number (default port number is 53533). If the domain has an internet proxy, specify the proxy information. Click ‘Apply’.
Keep the dialog open for the next step.

To deploy the Windows Feedback Forwarder group policy

The easiest way to configure machines in a domain to send CEIP data to your Windows Feedback Forwarder is to deploy a group policy. There are 2 options to deploy the group policy. You can either use the Windows Feedback Forwarder configuration dialog or you use the Group Policy Management Console to create and link the group policy object.

Use Windows Feedback Forwarder configuration dialog

In the Windows Feedback Forwarder configuration dialog select the group policy tab.
Enter the domain name that you want to deploy the group policy object to and click ‘Find’. Note: you may have to enter credentials at this step depending on the settings of the current user context.
After the list of organizational units is populated, select one or more organizational units.
Click the ‘Apply’ button

Manually create a group policy object

In the Windows Feedback Forwarder configuration dialog, select the ‘Forwarding Settings’ tab. Copy the Windows Feedback Forwarding URL and store it temporarily.
In GPMC create a new group policy object and set:

An alternative method to enable CEIP is the Windows Automatic Feedback dialog, which is a new multi-machine opt-in experience that ships in Server Manager. It enables you to configure multiple individual machines to send CEIP data within just 3 clicks.

Launch Server Manager and select ‘All Servers’ in the left hand navigation.
In the ‘Servers’ tile select ctrl+a to select all servers -> right click and select ‘Configure Windows Automatic Feedback’
Clicking Enable both Customer Experience Improvement Program and Windows Error Reporting will enable both on all servers connected to that Server Manager console

We would love to know what you think of this program and how we can improve it to provide the best experience for your deployments and Windows Server usage. Please give us your comments below.

Karen Albrecht
Program Manager
Windows Server Telemetry

Windows Server 2012 Remote Desktop Services (RDS)

2012-05-09T00:12:00Z

The other day I was in a conversation where I drew the distinction between reliable and robust. I hadn’t really thought about it precisely but when asked to articulate the distinction I said that robust was “reliable across a wide range of conditions”. A lot of what Klaas describes in his blog about RDS reminds me of that definition. Remote Desktop Services in Windows Server 2012, is reliable across a much wider range of conditions. It works better across a wide range of networking configurations, it works better across a wide range of hardware devices and configurations (physical or virtual) and it works better across a wide range of administrative scenarios. Oh yeah, it also adds a bunch of great new features. I think you are going to enjoy what you see here.

Klaas Langhout, a Director of Program Management in our RDS team, wrote this blog.

--Cheers! Jeffrey

For Windows Server 2012 we listened to our customers and partners and added the most desired features and resolved the top pain points in Remote Desktop Services (RDS). Following a description of RDS, I’ll summarize some of the many dramatic improvements we have made.

For those people that are not familiar with RDS, it is the workload within Windows Server that enables users to connect to virtual desktops, session-based desktops and RemoteApp programs. The key value that RDS provides is the ability to centralize and control the applications and data that employees need to perform their job from the variety of devices that the employee uses. This provides “work anywhere from any device” while ensuring that your control and compliance needs are met.

In the previous release, we received consistent feedback that:

RemoteFX was very popular however its underlying protocol (RDP) did not provide a great experience over Wide Area Networks (WANs)
Session and virtual machine infrastructures were complicated and costly and
The administration experience was not simple.

Windows Server 2012 addresses each of these issues.
For Windows Server 2012 we have made RemoteFX dramatically better over a WAN as well as balancing between scale (host side cost) and reduced bandwidth. Specific improvements include:

Adaptive Graphics. We support a mix and match approach, determining and using the right codec for the right content instead of one size fits all. We included codecs optimized for multimedia, images, and text. We improved caching as well as added progressive rendering. Progressive rendering allows RemoteFX to provide a responsive experience over a highly constrained network.
Intelligent Transports. We support UDP as well as TCP. UDP provides a better experience over a lossy WAN network but, is not always possible dependent on the routers, and firewalls involved. RDP will automatically use TCP when UDP cannot be used to ensure connectivity and the best possible experience.
Optimized Media Streaming. We utilize a new codec to reduce bandwidth consumption for media content (in some cases a 90% bandwidth reduction) while also providing a great end user media experience.
Adaptive Network Auto Detect. In this release, the end user no longer has to set the network in the Remote Desktop Connection client: the client auto-detects the network type and, also adapts as the network changes.
DirectX11 Support with vGPU. In Windows Server 2008 R2 SP1, we first introduced the RemoteFX Virtual GPU (vGPU), which provided DirectX 9 application support and Aero theming for virtual machines running on Hyper-V servers with physical GPUs. In Windows Server 2012, the vGPU feature is expanded and all Windows 8 virtual machines can take advantage of a DirectX 11 capable GPU, either emulated in software (softGPU) when no GPU is present in the host or para-virtualized and hardware-accelerated (vGPU) when a DirectX11 compatible video card is present in the host. We do support multiple GPU’s within one server and are seeing greater engagement with OEM’s to provide systems that support this.
Single Sign-On. In Windows Server 2008 R2, it was possible to configure an RDS deployment so that users will need to enter their credentials only once when connecting to RemoteApps and hosted desktops. However, this configuration was very cumbersome. In Windows Server 2012 we dramatically simplified this by eliminating the need to use multiple certificates. We also made it possible to use locally logged on domain credentials so that users connecting from managed devices can connect seamlessly without any credential prompts.
Email and web discovery of Remote Applications and desktops. Users now can find the correct remote workspace to connect to by just providing their email address. This removes the requirement to remember a long website URL. In addition, Remote Desktop Web Access now supports other browsers such as Chrome, Firefox, and Safari.
Multi Touch. We support full remoting of gestures (e.g. pinch and zoom) between the client and host with up to 256 touch points. This provides for a consistent experience when using a touch enabled device locally or, over RemoteFX. As more apps are written supporting touch as the primary interface, this will become more important.
USB Redirection. In Windows Server 2008 R2 SP1 we supported USB isochronous remoting only for vGPU enabled virtual machines. We have added support when using sessions and physical hosts which provides a consistent experience independent of physical, session, or virtual machine based host.
Metro-style Remote Desktop. In the app store we have added a new Metro-style application to provide an immersive touch-first remoting experience. Discoverability of remote resources, touch optimization, easy reconnect to your favorites, are just some of the specific features added.

The second main improvement area is in overall infrastructure simplification and cost reduction. Cost and complexity is a major roadblock for Virtual Desktop Infrastructure (VDI) and hosted desktop deployments of all sizes. In Windows Server 2012 we made many improvements to address this problem, such as:

Robust Pooled Virtual Desktop Collection model. “Pooled virtual desktop collection” model refers to the idea that a large number of virtual machines can be managed as a single entity by using a single virtual desktop template. This model is very attractive in VDI because it allows IT admins to provide a work desktop to multiple users without having to maintain a full OS for each user. In Windows Server 2012 we fully support this deployment model. Virtual machines can be created in batch from a virtual desktop template, patched by only modifying that virtual desktop template, and recreated/refreshed automatically by the RD Connection Broker. This dramatically reduces the cost and complexity of supporting a large number of users.
User Profile Disk. A major blocker for the “pooled virtual desktop collection” model has been lack of personalization: Since the pooled virtual desktop collection is based on a common virtual desktop template, the user’s personal documents, settings, and configurations would normally not be present. User Profile Desk was added to solve this problem for either virtual machine-based or session based desktop deployments. As the user logs on to different virtual machines within the pool or different RD Session Hosts within the session collection, his/her User Profile Disk gets mounted, providing access to the user’s complete profile. Since User Profile Disk operates at a lower layer, it works seamlessly with existing user state technologies such as Roaming User Profiles and Folder Redirection.
Wide range of high-performance and low cost storage options. RDS is built on top of Hyper-V and Windows Server 2012 storage, so the enhancements made throughout the hypervisor and storage stack in Windows Server 2012 benefit all RDS deployments. To name a few, we support:

VDI over SMB, SANs, or direct attached local storage
Pooled virtual desktop collections can be configured with storage tiers to optimize IOPS
Highly scalable and resilient configurations with Clustering and with Storage Spaces
All these improvements provide a dramatic reduction in costs while maintaining performance and management benefits of central storage.

Fairshare of resources in RD Session Host. In Windows Server 2012, RD Session Host server allocates CPU, Disk I/O, and Network I/O such that a single user cannot consume resources that would negatively impact other users on the same host. Each user will get a “fair share”. This is done with minimum overhead so the CPU, disk, and network resources are used to maximum capacity.
GPU Optional. In Windows Server 2008 R2 SP1 we had a requirement on a physical GPU for the new RemoteFX features that shipped in that release. In Windows Server 2012 the physical GPU is optional for VDI where it provides value if you are running applications that could benefit from hardware offload such as a CAD/CAM application.
Removal of a dedicated RD Session Host server running in redirection mode. We have removed the RD Session Host server running in Redirection mode which was a required component in previous versions. This functionality is now incorporated into the RD Connection Broker. This reduces the number of components to deploy and manage.

The third and final focus area for improvements made in RDS has been in overall management simplification. This is targeted at improving the E2E management experience as well as enabling partner solution creation. Improvements include:

RDS Management Interface integrated into Server Manager. RDS now includes a single management interface through which you can deploy RDS end to end, monitor the deployment, configure options, and manage all your RDS components and servers. This management interface is built into the new Server Manager, taking advantage of many new Windows Server 2012 management capabilities such as multi-server deployments, remote configuration, and orchestrated configuration workflows. This interface replaces older tools such as Remote Desktop Services Manager, RemoteApp Manager, and RD Session Host Configuration. The management tools for RD Gateway and RD Licensing are still provided separately since these roles are often deployed independently.
Scenario-Focused Deployment. The new Server Manager provides a scenario-focused wizard that dramatically simplifies the task of bringing up a complete RDS deployment. This wizard sets up all the roles needed for an RDS deployment, configures each server role correctly to communicate with the other roles, and walks you through creating your first virtual desktop or session collection as well. The wizard comes in two flavors:

Quick Start is optimized for deploying Remote Desktop Services on one server, and creates a collection and publishes RemoteApp programs.
Standard Deployment allows you to deploy Remote Desktop Services across multiple servers, allowing for a more customized deployment.

Active/Active RD Connection Broker. In previous releases the RD Connection Broker role service has supported an active/passive clustering model. This provided high availability in the case of component failure, but it did not address high scale requirements. In this release, we have eliminated the need for clustering and switched to an active/active model. With this model, two or more RD Connection Brokers can be combined as a farm to provide both fault tolerance and load balancing. This prevents the broker from being a single point of failure and also allows ‘scale out’ as load demands.
PowerShell support. All platform functions and capabilities can be controlled through a comprehensive and rich PowerShell layer. IT administrators can use this layer to build sophisticated automation that helps fit RDS into their IT infrastructure and workflows. We also anticipate third-party vendors to use this new extensibility layer to address unique new scenarios and integrate Windows Server 2012 RDS into management tools.

Remote Desktop Services in Windows Server 2012 provides a single infrastructure, and consistently great remoting experience even over WAN while offering three deployment choices: Session, Pooled virtual desktop collection, Personal virtual desktop collection to reduce the cost appropriate to the needs of the user. The administration is simplified and platform hooks are provided for partner extension to provide additional value and solutions.

Customers are excited about RDS with Windows Server 2012 and some have already rolled out a pre-release version into production taking advantage of these new benefits! We are proud of the work we have done and look forward to providing more information as we drill into the specific features in blogs posts to come at the RDS Blog.

- The Entire Remote Desktop Virtualization Team

Introducing the Server and Cloud Partner and Customer Solutions Team Blog

2012-05-07T18:44:00Z

You’ve heard me talk time and time again about how much time we spent with customers and partners during the planning of Windows Server 2012. Well we didn’t stop at the end of planning. In today’s blog, Natalia introduces herself, her team and their blog which will document some of the details of our customer/partner engagements and how they are succeeding with Window Server 2012.

--Cheers! Jeffrey Snover

Hi, my name is Natalia Mackevicius, the Group Program Manager of the Windows Server Partner and Customer Ecosystem. Back in October, I covered my role and outlined how my team has applied the all-important voices of customers and partners to the planning and development of Windows Server 2012: Windows Server 8: Driven by the Voice of the Customer and Partner

Today, I’m pleased to introduce the Server and Cloud Partner and Customer Solutions Team Blog. Over the coming months, this blog will be used to share examples of how our customers are implementing the new capabilities of Windows Server 2012 and highlight opportunities for partners to develop hardware and software for Windows Sever 2012. These examples will be tied back to the Windows Server Blog and the Product Team blogs for more detailed technical information on the features themselves.

Here are a few things we are going to include in our blog:

Discuss how early adopter customers are using Windows Server 2012 technologies to solve current pain points and/or address new opportunities and scenarios
Provide early information on customer adoption and best practices from the Engineering validation programs we run.
Discuss the Windows Server 2012 scenarios and the technical integration points for partners, such as API’s, hardware and applications
Provide an overview of the reference architectures or types of technologies that are being developed by partners to complete and/or extend Windows Server 2012 solutions, as they become publically available.

We hope that you find this blog useful. We will try to provide a regular rollup of what we have covered on the Server and Cloud Partner and Customer Solutions Team Blog from here so that you don’t miss anything.

We welcome your comments and questions. Join the discussion!

Please let me introduce to my team’s first blog post: Windows Server 2012 Technology Adoption Program - TAP

Natalia Mackevicius
Group Program Manager
Partner and Customer Ecosystem

Building Cloud Infrastructure with Windows Server 2012 and System Center 2012 SP1

2012-05-03T18:15:00Z

Operating Systems are platforms delivering experiences, features, and APIs that developers can build upon. Today, many developers take already shipping versions of Windows and deliver cloud computing solutions. Windows Server 2012 is a cloud-optimized OS, which means that developers can deliver much better cloud computing solutions with much less effort. System Center 2012 already delivers great cloud computing solutions using Windows Sever 2008/R2. In this blog, Anders Vinberg, a Technical Fellow in our Management Division, describes how the Virtual Machine Manager component in System Center 2012 SP1, now available as a community technology preview, builds on the cloud optimizing features of Windows Server 2012 to take that solution to the next level.

--Cheers! Jeffrey

With the official naming of Windows Server “8” as Windows Server 2012 and the launch of System Center 2012 at MMS a few weeks back, Microsoft has now delivered a solution to our customers for building their private clouds and to hosters for building their own Infrastructure-as-a-Service public cloud offerings. It is instructive to recap the meaning of moving to the cloud model and the core tenets of a cloud as was laid out in the keynote by Brad Anderson at MMS, and then take a look at how this is done with Windows Server 2012 and System Center 2012 SP1.

The Cloud Model
First off, it is important to note that cloud computing does not necessarily mean that the workload is running outside a customer’s premises. The workloads could be deployed on infrastructure that is on a customer’s premises, or on their partners’ premises but completely controlled and managed by the customer. That is a “private” cloud. Workloads could also be deployed and run on a hoster’s premises on shared infrastructure that is used by other tenants. That is a “public” cloud. In both cases, cloud computing is a way of consuming capacity with the attributes of resource pooling, self-service, elasticity and usage-based metering.

The Cloud Personas
As the cloud model decouples the infrastructure from the services it supports, it also decouples two distinct processes: provisioning and consumption. And there are two corresponding personas:

Service provider (the datacenter admin)
Service consumer (an application owner)

These two personas look for quite distinct attributes, each in their domain:

The separation of concerns between the provider and consumer offers great simplicity and agility. It is a foundation for the trend toward democratization of computing. We often hear that the consumer should not have to be aware of the details of the physical infrastructure, but we can make a stronger statement: the consumer is not allowed to be aware of the physical infrastructure, because that would constrain the daily work of the provider. The provider may need to replace an old machine with a new one that is more efficient, and should not have to involve or even inform the consumer, as long as the abstractions and service level agreements are satisfied. This decoupled model does not fit with all existing IT processes or with all existing apps; in a coming blog we will discuss how Windows Server and System Center accommodate a mix of work styles.

Cloud Attributes Realized
Let’s look at each of the four cloud attributes and see what Windows Server 2012 and System Center 2012 provide customers.

Pooled resources: This means that we deal with resources at an aggregate level rather than at the level of individual servers. The cloud exposes a pool of capacity for use by services that require the capacity, and this abstraction decouples the virtualized workloads from the physical infrastructure, allowing dynamic workload placement and independent infrastructure management.
While modern large-scale clouds often use strictly homogeneous hardware and require that software adapts, this is often not practical in enterprise computing where existing software may have specific hardware requirements; our cloud model supports heterogeneous resource pools, where the system automatically matches software requirements to hardware characteristics.
Having pools of resources implies that multiple tenants (customers) will have their workloads on this environment and the infrastructure must provide the necessary isolation between fenced-off resource pools. Such multi-tenancy is not just for public clouds: even in a private cloud, the self-service model that gives consumers flexibility to deploy services with little oversight requires robust isolation between pools to prevent accidental impact on a neighbor.

Windows Server 2012 enables resources to be pooled via a variety of capabilities such as the Hyper-V extensible switch, Network Virtualization, Quality of Service (QoS) and network isolation policies. In addition, with enhancements in live and storage migration, the Windows Server platform enables resources to be moved easily across the datacenter, to optimize the use of datacenter resources.
System Center 2012 through the Virtual Machine Manager component can aggregate compute, network and storage resources and expose them as a construct called a “Cloud”. It supports managing these Clouds at scale, and dynamically placing workloads in them, with role-based access control mechanisms for multi-tenant isolation and delegation of clouds to consumers. In SP1, Virtual Machine Manager uses the platform capabilities of network virtualization and live and storage migration for more flexible pool management and to load-balance the environment so that customers SLA’s are met proactively.

Self-Service: In the cloud model, service consumers can use a self-service experience, typically a web-based portal, to access the capacity they have been allocated, self-provision workloads from standing up a single VM to deploying a complex service, and manage the life cycle of those workloads.

Windows Server 2012 goes a long way in enabling full datacenter automation. Self-servicing implies that all datacenter operations must be fully automatable, otherwise manual labor will be required every time a workload is placed on a cloud. Windows Server 2012 is fully automatable via PowerShell and WMI, exposing the necessary interfaces to enable this scenario.
System Center 2012 builds on the automation capabilities in Windows Server 2012 and provides portals and management capabilities to enable self-service. The Service Manager component provides a service catalog that drives a self-service portal for IT approval workflows such as allocating capacity. The App Controller component provides a self-service experience for administering virtual machines and services, covering both private cloud and the Windows Azure public cloud. The Operations Manager component provides the operational intelligence for the environment, and the Orchestrator component provides run-book automation. Lastly, the Data Protection Manager component of System Center implements business continuity policies.

Elastic: Cloud Elasticity means that the infrastructure can support the changing needs of the organization, deploying new services as needed, allocating more resources to services that experience heavy load or de-allocating resources to save power when the load is light. With cross-cloud management, workloads can also move between private and public clouds, providing extra capacity, geo-scale reach, or other characteristics as needed.

From the Windows Server platform perspective, elasticity is enabled by allowing multiple services running on different infrastructures to be interconnected via IPSec VPNs. Windows Server 2012 has new support for IKEv2 VPNs in the box, allowing it to easily interconnect private and public clouds.
In addition, elasticity also means that it should be possible to easily move any workload across the cloud to public cloud providers. In current technologies, this is very hard to achieve because workloads tend to have a lot of networking assumptions embedded into them, such as fixed IP addresses and subnets. With Windows Server 2012 Network Virtualization, it is now possible to move a workload around while keeping its own IP addresses and decoupling it from the provider’s IP space.
System Center 2012 SP1 uses a platform capability for network virtualization in its network constructs. When a workload “network” is defined, System Center allows cloud consumers to deploy such networks on any cloud or on any physical network infrastructure that is made available to them.
VMM not only allows elastic allocation and release of resources to services within a cloud, but also allows adding or removing capacity to the cloud itself, giving the appearance of unlimited capacity of the cloud as viewed by the service consumer.

Usage Based: In the cloud model, customers are billed or at least get informed on their cloud resource usage based on their actual resource consumption.

Windows Server 2012 provides capabilities for detailed and granular metering information for core metrics such as CPU, memory, storage and network. In Windows Server 2012, these metrics follow the VM as it migrates in the environment.
System Center 2012 aggregates these consumption metrics and allows the cloud operator to show back or bill back based on their policies.

A detailed walkthrough of the various features and capabilities that make Windows Server 2012 a cloud-optimized OS can be found in the white paper Building an Infrastructure as a Service (IaaS) Cloud Using Windows Server 8.

Scenarios
As we can see from above, there are many aspects of a cloud. In this blog we will focus on the Service provider persona and specifically on how providers can stand up their private cloud infrastructure as it pertains to using SMB 3.0 as storage for VMs and using Hyper-V Network Virtualization with Windows Server 2012 Beta and the community technology preview (CTP) of System Center 2012 SP1 Virtual Machine Manager (VMM). In future posts we will delve deeper into the other aspects of the cloud.

Standing up Cloud Infrastructure with System Center 2012 SP1
Let’s start by looking at how Hyper-V network virtualization is provisioned and managed from VMM. In System Center 2012, VMM introduced Logical Networks which abstracts the various definitions of networks in enterprise datacenters, allowing datacenter administrators to use the vernacular of the application owners who express their connectivity using terms as “I want my VM to connect to the CORP network”. A logical network could be defined differently for each datacenter site and automation in VMM ensures that when the VM is deployed the appropriate configuration is applied. With SP1, we introduce another abstraction over this called “VM networks”. Logical networks now pertain to the fabric networks and VMs and Services now only connect to “VM Networks”. A VM network can be realized by a VLAN, direct logical network or with Windows Server 2012 with Hyper-V Network Virtualization.
In the System Center 2012 SP1 CTP VMM only supports creating VM networks with Hyper-V network virtualization using Generic Routing Encapsulation (GRE) which is the long term preferred mechanism. In the final release of System Center 2012 SP1, we plan to support creating VM networks using IP Rewrite which is easier to deploy in existing environment and doesn’t require a change of network infrastructure, but does require a provider address (PA) for each customer address (CA) you allocate. I strongly urge you to read the great blog on Hyper-V Virtual Networking to get an understanding of how this technology works.
The PAs are allocated from the Logical network space so you should create a Logical network as you did previously and allocate an IP address pool from which VMM can pull addresses for the PA space. Next you need to create a VM Network, which is the network that will be used by the actual services being deployed. VM networks can be created with just a few clicks from the new node in the VM’s and Services view in the VMM console. A detailed step-by-step guide for this can be found here.

In the example above you can see that both the Tailspin network and the Wingtip Network have overlapping IP ranges. They are realized and automatically provisioned using Hyper-V Network virtualization, providing full isolation without any special hardware or additional software. When creating a VM, it can now be connected to this VM network, thereby allowing it connectivity to other VM’s on the same VM network, while keeping the VM isolated from other VM networks that belong to different customers even though they are using the same subnet.
For service providers who need to provide isolated environments to their service consumers (tenants), this capability is invaluable and provides the flexibility to enable the tenants to bring their own IP addresses to the public cloud environment. In the CTP, if you want the VM on a VM network to communicate with entities not on the VM network you will need to set up a gateway between these networks. This can be done using a Windows Server instance with the appropriate routing rules and you can expect a future guide to walk you through the process of how to set it up. In addition, System Center will allow this to be done seamlessly as we move forward with development.
Storage is another vital component of a cloud and virtualization project. With Windows Server 2012 we now have the ability to use SMB 3.0 file shares for hosting Hyper-V VM’s in a clustered and standalone environment. This helps drive the cost of cloud down while adding flexibility and making management easier. (You can read more about storage for cloud here.) System Center 2012 SP1 makes it very easy to use. The screen shots below depict how you can add a file share as storage for a cluster and for a standalone host, and VMM configures the Access Control Lists appropriately for this configuration.

Standalone Host

Hyper-V Cluster

Once a VM is deployed onto a host and particular storage sub-system, the service provider desires flexibility to move the workload to different hosts or to use different storage to ensure that VMs are up even when the host needs to be serviced or the storage environment needs to be maintained. With Windows Server 2012 and VMM we now offer multiple options for live migrating the VM and its associated storage. You can:

Live migrate the VM within a cluster (which normally has shared block or file storage)
Live migrate the VM in and out of a cluster
Live migrate the storage of the VM from one storage sub-system to the other
Live migrate the VM from one host to the other (with no shared storage)

Just imagine the flexibility that this provides you as a datacenter administrator. The screenshot below depicts these various options from within VMM.

As you can see on the left side of the above screenshot, a VM called Tailspin_VM2 runs on a standalone host HV104. The dialog on the right shows that it can be migrated from this standalone host into nodes of the HVClusterA cluster (hv103n3, hv101n1 and hv102n2) as well as to the standalone HV105. System Center automatically detects there is no shared storage between HV104 and HVClusterA and tags these migrations as “Live (VSM)”, indicating that storage would be migrated too, and not just the virtual machines.

Note that System Center also gives you the option to storage migrate the VM’s storage within the host with no downtime for the VM. This is useful if for example you are running out of local storage on a particular drive and want to move the VM’s storage onto a different drive with more capacity on the host.

Now the perceptive would have noticed that we show only “Live” to HV105! Why is that? No it’s not a bug. To get an understanding of that let’s take a look at the storage property for HV104 (the host the VM is currently on) and HV105. As you will notice, each of these hosts see the same SMB 3.0 share and hence VMM can migrate the VM (without having to move the storage).

Summary
In this blog we discussed the cloud model and the two different cloud personas (“Service provider” and “Service Consumer”). We also described how Windows Server 2012 and System Center 2012 SP1 deliver this model. We highlighted how Windows Server 2012 and the Virtual Machine Manager component in System Center 2012 SP1 provide the ability for service providers to utilize SMB 3.0 storage for VM’s and create isolated networks using Hyper-V Network Virtualization. Over the next few months we will provide additional details of how VMM can facilitate resource pooling and tenant administration, and how it can utilize the plethora of capabilities in Windows Server 2012.

Windows Server 2012 Release Candidate Timing

2012-04-24T17:15:01Z

In my last blog, I mentioned that as we progress towards releasing the next version of Windows, many of the details are getting decided and communicated. Last week we announced our official product name (“Windows Server 2012”) and that the final product will be delivered this year. Yesterday Steven Sinofsky delivered the keynote at “Windows Developer Days” in Tokyo, Japan where he announced that the Windows 8 Release Preview will be publicly available in the first week of June 2012. You can see the official MSFTNews tweet at: http://twitter.com/#!/MSFTnews/statuses/194831068422094848 . We plan to deliver a release candidate of Windows Server 2012 in the same timeframe.

--Cheers! Jeffrey

SMB 2.2 is now SMB 3.0

2012-04-19T21:06:00Z

We are at an exciting stage of the release. Beta has been out for some time now and we’ve gotten lots of great feedback. As we progress towards releasing the next version of Windows, many of the details are getting decided and communicated. This has been a big week for naming. On Monday, Brandon LeBlanc announced the official product names for editions of Windows on the Windows Team blog. During that day’s MMS keynote, Brad Anderson announced the official name for Server: Windows Server 2012. In today’s blog the SMB team announces their official name: SMB 3.0. I don’t think this will come as much of a surprise to anyone. The team has delivered an amazing amount of innovation in this release. If you haven’t already downloaded the beta, I think you’ll want to after reading some of the details in this blog.

-Cheers! Jeffrey

Last September at the //Build Conference, we announced SMB 2.2, an update to our Server Message Block protocol used by default for file sharing in Windows. Since then we have actively engaged with the community through various channels and have spoken in detail about all the great work that has gone into the release and why we think this is truly a game changer. Windows Server 2012 provides a vast set of new SMB features with an updated SMB protocol that greatly enhance the reliability, availability, manageability, and performance of file servers.

Looking back at the amount of changes that have gone into this release – the lines of code written, array of features introduced, new scenarios we have enabled, work we have done with our partners, a minor revision doesn’t do justice the work that has gone in. So moving on, SMB 2.2 is SMB 3.0!

Regular followers of this blog have seen detailed posts on various SMB improvements over the last few months. To summarize, the following are some of the key new functionalities available with Windows Server 2012 SMB 3.0:

SMB for Server Applications – Many of the new SMB features are specifically designed for server applications that store the data on file shares—for example, database applications such as Microsoft SQL Server or virtualization software such as Hyper-V. This allows applications to take advantage of advances in storage management, performance, reliability, and cost efficiency that come with SMB to deliver an application storage solution that rivals traditional Fibre Channel storage solutions in features and capabilities, but remains easier to provision and less expensive to implement.
Active file sharing with SMB Scale Out – Enables customers to scale share bandwidth by adding cluster nodes, as the maximum share bandwidth is the aggregate bandwidth of all file server nodes and not restricted to the bandwidth of a single cluster node as in previous versions. Scale-out file shares also makes it much easier to manage a file server cluster, as it is no longer necessary to create multiple clustered file servers, each with separate cluster disks, to take advantage of all nodes in a cluster. Further, the administrator can transparently redirect SMB client connections to a different file server cluster node to better balance the cluster load.
Scalable, fast, and efficient storage access with SMB Direct – SMB Direct (SMB over Remote Direct Memory Access (RDMA)) is a new transport protocol for SMB in Windows Server 2012. It enables direct memory-to-memory data transfers between servers, with minimal CPU utilization and low latency, using standard RDMA-capable network adapters (iWARP, InfiniBand, and RoCE). Any application which accesses files over SMB can transparently benefit from SMB Direct. Minimizing the CPU cost of file I/O means application servers can handle larger compute workloads with the saved CPU cycles (for example, Hyper-V can host more virtual machines).
Fast data transfers and network fault tolerance with SMB Multichannel – Given that customers can now store server application data on remote SMB file shares, SMB was enhanced to improve network performance and reliability. SMB Multichannel takes advantage of multiple network interfaces to provide both high performance through bandwidth aggregation, and network fault tolerance through the use of multiple network paths to data on an SMB share.
Transparent Failover and node fault tolerance with SMB – Supporting business critical server application workloads requires the connection to the storage back end to be continuously available. The new SMB server and client cooperate to make failover of file server cluster nodes transparent to applications, for all file operations, and for both planned cluster resource moves and unplanned node failures.
VSS for SMB file shares – VSS for SMB file shares extends the Windows Volume ShadowCopy Service infrastructure to enable application-consistent shadow copies of server application data stored on SMB file shares, for backup and restore purposes. In addition, VSS for SMB file shares enables backup applications to read the backup data directly from a shadow copy file share rather than involving the application server in the data transfer. Because this feature leverages the existing VSS infrastructure, it is easy to integrate with existing VSS-aware backup software and VSS-aware applications like Hyper-V.
Secure data transfer with SMB encryption – SMB Encryption protects data in-flight from eavesdropping and tampering attacks. Deployment is as simple as checking a box, with no additional setup requirements. This becomes more critical as mobile workers access data in centralized remote locations from unsecured networks. SMB Encryption is beneficial even within a secured corporate network if the data being accessed is sensitive..
Faster access to documents over high latency networks with SMB Directory Leasing – SMB Directory Leasing reduces the latency seen by branch office users accessing files over high latency WAN networks. This is accomplished by enabling the client to cache directory and file meta-data in a consistent manner for longer periods, thereby reducing the associated round-trips to fetch the metadata from the server. This results in faster application response times for branch office users
SMB Ecosystem – A critical aspect of Windows Server 2012 development is the partnership we have established with vendors to ship SMB 3.0 capable systems. We have been working closely with several server vendors and open source partners over the past year, by proactively providing extensive protocol documentation and numerous open “plugfest” events provide opportunities for test and feedback. Finally, and most importantly, the SMB ecosystem now reaches all the way to key server applications such as SQL Server and Hyper-V to ensure that SMB 3.0 capabilities are fully leveraged all the way through the stack, and across the multivendor network.

With so many new features, SMB offers a richer set of capabilities that, when combined, provide organizations with a robust high performance storage alternative to traditional Fibre Channel storage solutions at a much more affordable cost point from both an acquisition and operational perspective.

For additional reading materials related to SMB, there is a ton of content on TechNet:

- Windows Server “8” Beta SMB Overview
- High-Performance, Continuously Available File Share Storage for Server Applications Technical Preview
- Deploying Fast and Efficient File Servers for Server Applications
- Building Your Cloud Infrastructure: Converged Data Center with File Server Storage

Jose Barreto and Claus Joergensen, Program Managers in the SMB team, have written a great blog on taking server application storage to Windows file shares. It’s a great read and is highly recommended.

We are really excited about Windows Server 2012 and can’t wait for you to start using the product. We definitely hope you have as much fun using the product as we had building it. In the meantime, as you know, the Beta has been out for a while now and we would love for you to use it and let us know what you think.

SMB 3.0 Team.

Introducing Windows Server “8” Hyper-V Network Virtualization: Enabling Rapid Migration and Workload Isolation in the Cloud

2012-04-16T20:14:00Z

We’ve all heard about the agility that server virtualization delivers. However, our conversations with people in the trenches made it clear that the full potential of virtualization remains frustratingly beyond their grasp. In particular, the lack of agile networking limits the agility you can achieve at a reasonable cost.

Windows Server “8” is the most cloud optimized operating system, providing choice and flexibility in configuring private, hybrid, and public cloud solutions. Bill Laing, in his blog post, Windows Server “8” Beta Available Now, outlined some of our key investments, including Hyper-V Network Virtualization. In this blog post, Sandeep Singhal (General Manager of the Windows Networking team) and Ross Ortega (Principal Program Manager from the Windows Networking team) describes some of the issues surrounding cloud adoption and how Hyper-V Network Virtualization in Windows Server “8” addresses these challenges.

Cheers – Jeffrey Snover

We’ve spent the past couple of years talking with customers about why they haven’t yet deployed their workloads to a cloud. We consistently heard three main issues. First, they want to gradually begin moving individual services to the cloud with a flexible hybrid cloud solution. Second, moving to the cloud is difficult. It’s tedious, time-consuming, manual, and error-prone. Third, customers express concern about their ability to move to the cloud while preserving isolation from other tenants, be they other business units in the private cloud or competitors in a public cloud. In the end, whether you’re building your own private clouds or considering using a public cloud provider, you want easy onboarding, flexibility to place your virtual machines anywhere—either inside or outside the cloud, and workload isolation.

Network Agility: An Unfulfilled Promise
Underlying all of these concerns, customers want the control and flexibility to move their services to the cloud, move them to a different cloud provider, or even move them back to their enterprise datacenter. However, today this is quite labor intensive because cloud hosters require that their customers change the IP addresses of services when those services are moved to a particular cloud environment. This seems like a minor deployment detail, but it turns out that an IP address is not just some arbitrary number assigned by the networking folks for addressing. The IP address also has real semantic meaning to an enterprise. A multitude of network, security, compliance, and performance policies incorporate and are dependent on the actual IP address of a given service. Moving to the cloud means having to rewrite all these policies. Of course you have to find them all first and then negotiate and coordinate with the different organizations that control those policies. If you wanted to move to a different cloud provider then that new hoster would assign different IP addresses, requiring yet another policy rewrite. The current situation blocks many customers and scenarios from adopting the cloud.

Customers asked us to have Windows make it appear that their services in the cloud were similar to the services running in their internal datacenters, while adhering to their existing policies and providing isolation from other VMs running in the cloud hosting environment. When moving to the cloud customers want their data to be as isolated and as safe as if it were running in their own datacenter.

In summary, you demanded the ability to run Any Service on Any Server in Any Cloud.

We took this feedback seriously and designed a new technology called Hyper-V Network Virtualization in Windows Server “8” to provide a scalable, secure multi-tenant solution for those building cloud datacenters and to make it easier for customers to incrementally move their network infrastructure to private, hybrid, or public clouds. As we will describe later, Hyper-V Network Virtualization builds on existing IETF and IEEE standards, providing interoperability with existing and future network equipment, security appliances, and operational processes.

Hyper-V Network Virtualization: Applying Server Virtualization to Entire Networks
With traditional server virtualization, each physical host is converted to a virtual machine (VM), which can now run on top of a common physical host. Each VM has the illusion that it is running on a dedicated piece of hardware, even though all resources—memory, CPU, and hardware peripherals are actually shared.

Network virtualization extends the concept of server virtualization to apply to entire networks. With network virtualization, each physical network is converted to a virtual network, which can now run on top of a common physical network. Each virtual network has the illusion that it is running on a dedicated network, even though all resources—IP addresses, switching, and routing—are actually shared.

Hyper-V Network Virtualization allow customers to keep their own internal IP addresses when moving to the cloud while providing isolation from other customers’ VMs – even if those VMs happen to use the exact same IP addresses. We do this by giving each VM two IP addresses. One IP address, the IP address visible in the VM, is relevant in the context of a given tenant’s virtual subnet. Following the IEEE nomenclature we call this the Customer Address (CA). The other IP address is relevant in the context of the physical network in the cloud datacenter. This is called the Provider Address (PA). This decoupling of tenant and datacenter IP addresses provides many benefits.

The first benefit is that you can move your VMs to the cloud without modifying the VM’s network configuration and without worrying about what else (or who else) is sitting in that datacenter. Your services will continue to just work. In the video demo referenced at the end of this article we used traceroute, a low-level network diagnostic tool, to show how on-premise services were interacting transparently with services that had been moved to the cloud. We highlighted the fact that once the services moved to the cloud, packets simply were now taking an extra hop to get to the cloud datacenter. The virtual subnet has become a nearly transparent extension of the enterprise’s datacenter. We also created a secure encrypted tunnel to the virtual subnet. The end result is that different customers with the exact same IP address connected to the same virtual switch are isolated.

Imagine Red VM having IP address 10.1.1.7 and Blue VM having 10.1.1.7 as shown above. In this example the 10.1.1.7 IP addresses are CA IP addresses. By assigning these VMs different PA IP addresses (e.g. Blue PA = 192.168.1.10 and Red PA = 192.168.1.11) there is no routing ambiguity. Via policy we restrict the Red VMs to interact only with other Red VMs and similarly Blue VMs are isolated to the Blue virtual network. The Red VM and the Blue VM, each having a CA of 10.1.1.7, can safely coexist on the same Hyper-V virtual switch and in the same cloud datacenter.

Second, policy enforcement in the end hosts provides a scalable solution for multi-tenant isolation. We do not need to reconfigure the network infrastructure to isolate tenants from each other. Before Hyper-V Network Virtualization, the common solution was to use VLANs for isolation. However, VLANs have scalability limitations, only supporting a limited number of tenants in a shared datacenter. In addition to having scalability limitations, VLANs are more suited for static network topologies and not the more dynamic environment in which tenants may continually join and leave the cloud datacenter or tenant workloads may continually be migrated across physical servers for load balancing or capacity management purposes. VLANs require the reconfiguration of production switches every time a VM needs to be brought up on a new server. Typically, the VM deployment team creates a service ticket to the network operations team to reconfigure the appropriate switches with the relevant VLAN tags. By eliminating this step, Hyper-V Network Virtualization increases the overall operational efficiency of running a datacenter.

Third, by allowing you to preserve your IP addresses when moving to the cloud, Hyper-V Network Virtualization also enables cross-subnet live migration. When we talk about live migration, we mean that any client talking to a service is unaware that the VM hosting the service has moved from one physical host to a different physical host. Previously cross-subnet live migration was impossible because, by definition, if you move a VM from one subnet to a different subnet its IP address must change. Changing the IP address causes a service interruption. However, if a VM has two IP addresses, then the IP address relevant in the context of the datacenter (Physical Address) can be changed without needing to change the IP address in the VM (Customer Address). Therefore the client talking to the VM via the CA is unaware that the VM has physically moved to a different subnet.

What’s really exciting is that cross-subnet live migration enables new scenarios. Recall our “Any Service, Any Server, Any Cloud” vision. VMs can now run and live migrate anywhere in the datacenter without a service interruption. New datacenter efficiencies can be achieved. For instance hosters, during light load periods (such as around 3am) can consolidate any active VMs to a subset of the datacenter and power off other parts of the datacenter—all without having to reconfigure the physical network topology. Administrators no longer need to worry about a VM being trapped in one part of the datacenter because its IP address physically restricts where that IP address is valid. Similarly VM deployment algorithms are free to assign VMs anywhere in the datacenter because the PA address relevant in the context of the physical datacenter can be changed independently of the CA address which is relevant in the context of the virtual network.

With Hyper-V Network Virtualization the virtual machine is totally unaware that its IP address is being virtualized. From the VM’s perspective, all communication is occurring via the CA IP address. Because the VMs are unaware that they are part of a virtual network, any operating system running within a Hyper-V VM (e.g. Windows Server 2008 R2, Windows Server 2003, Linux, etc.) can be a member of a virtual network. Hyper-V Network Virtualization is completely transparent to the guest OS.

Two Mechanisms for Virtualizing IP Addresses on a Subnet
Customers can deploy Hyper-V Network Virtualization in their existing datacenters using either IP virtualization mechanism without requiring any hardware upgrades or topology changes. We virtualize the CA IP address by using the PA when sending networking traffic between different end hosts. We use two different mechanisms to virtualize the IP address: Generic Routing Encapsulation (GRE) and IP Rewrite. For most environments, GRE should be used for network virtualization, because it provides the most flexibility and performance. However, IP Rewrite may be appropriate to provide performance and compatibility in some current high-capacity datacenters.

Within the source and destination Hypervisors, packets are associated with a Virtual Subnet ID. The Virtual Subnet ID allows the hypervisor to differentiate traffic from different virtual subnets that may share the same CA IP address (e.g., differentiating Red 10.1.1.7 from Blue 10.1.1.7). Using the Virtual Subnet ID, the Hypervisor can apply additional per-tenant policies, such as access controls.

The first IP virtualization mechanism is Generic Routing Encapsulation (GRE), an established IETF standard. In this case we encapsulate the VM’s packet (using CA IP addresses) inside another packet (using PA IP addresses). The header of this new packet also contains a copy of the Virtual Subnet ID. A key advantage of GRE is that because the Virtual Subnet ID is included in the packet, network equipment can apply per-tenant policies on the packets, enabling efficient traffic metering, traffic shaping, and intrusion detection. Another key advantage of GRE is that all the VMs residing on a given end host can share the same PA because the Virtual Subnet ID can be used to differentiate the various IP addresses from different virtual subnets. Sharing the PA has a big impact on scalability. The number of IP and MAC addresses that need to be learned by the network infrastructure can be substantially reduced. For instance, if every end host has an average of 20 VMs then the number of IP and MAC addresses that need to be learned by the networking infrastructure is reduced by a factor of 20. A current drawback of GRE is that the NIC offloads no longer provide the scalability benefit to the end host because the NIC offloads are operating on the outer header and not the inner header. The offloads can be important for high performance environments where a VM requires 10 gigabit bandwidth. Similarly entropy for datacenter multi-path routing is reduced because the switches, by hashing fields in only the outer packet, will not differentiate traffic coming from different VMs residing on the same end host.

Never fear! We have a solution for these limitations.

In Windows Server “8” we’ve made working with standards a high priority. Along with key industry thought leaders (Arista, Broadcom, Dell, Emulex, HP, and Intel) we published an informational draft RFC (NVGRE) discussing the use of GRE, an existing IETF standard, as an encapsulation protocol for network virtualization. Together with server, switch, and NIC partners we have demonstrated broad ecosystem support for Hyper-V Network Virtualization. Once our partners incorporate NVGRE into their products, hosters will get the scalability benefits of GRE without performance loss. They will also see opportunities to deploy multi-tenant-aware network equipment, including load balancers, firewalls, storage controllers, network monitoring and analysis tools, and other security and performance products.

GRE is the preferred network virtualization approach for most current and future datacenters. However, some current datacenters may need greater scalability than can be achieved with current generation hardware. For these environments, Windows Server “8” supports a second IP virtualization mechanism, IP Rewrite.

With IP Rewrite, we rewrite the source and destination CA IP addresses in the packet with the appropriate PA addresses as packets leave the end host. Similarly, when virtual subnet packets enter the end host the PA IP addresses are rewritten with appropriate CA addresses. A key advantage of IP Rewrite is that the packet format is not changed. Existing network hardware offload technologies such as Large Send Offload (LSO) and Virtual Machine Queue (VMQ) work as expected. These offloads provide significant benefit for network intensive scenarios in a 10 Gigabit Ethernet environment. In addition, IP Rewrite is fully compatible with existing network equipment, which does not see any new traffic types or formats. Of course, the Virtual Subnet ID is not transmitted on the network, so that existing network equipment cannot perform per-tenant packet processing.

An Incremental Approach to Creating Hybrid Clouds
With Hyper-V Network Virtualization we’ve made it easy to move your subnets to the cloud. However, once in the cloud the next thing you need is for your virtual subnets to interact with each other. For example, the typical 3-tier architecture is composed of a front end tier, business logic tier, and a database tier. You need a way for these virtual subnets (tiers in this example) to communicate as if they were all located in your own datacenter. Hyper-V Network Virtualization allows you to route between your virtual subnets. That is, not only can you bring your virtual subnet to the cloud, you can also bring your entire network topology to the cloud.

Windows Server “8” also provides a Cloud Cross-Premise connectivity solution that can securely connect your datacenter or private cloud with a public cloud to create a hybrid cloud. Combining Hyper-V Network Virtualization with Cloud Cross-Premise Connectivity means we have made the cloud a seamless extension of your datacenter.

Internally at Microsoft, we use Hyper-V Network Virtualization in a private cloud deployment using GRE as the IP virtualization mechanism. Here the tenants are the various product groups in the Server and Tools Business unit (STB). We wanted to consolidate our datacenter infrastructure to realize the operational and resource efficiencies of the cloud as well as providing our product groups the necessary flexibility they required when deploying their services in a cloud environment.

Conclusion
We’re excited about Hyper-V Network Virtualization because it benefits customers moving to private, hybrid, and public clouds; provides new efficiencies for hosters and administrators running cloud datacenters; and presents new opportunities for our ecosystem partners. Hyper-V Network Virtualization—combined with other technologies such as Storage Live Migration, simultaneous Live Migration, and Failover Replication—enables complete VM mobility with Windows Server “8.”

To learn more about Hyper-V Network Virtualization watch the demo we gave at //BUILD/. Our demo starts at 13 minutes and 52 seconds. Our //BUILD talk: Building secure, scalable multi-tenant clouds using Hyper-V Network Virtualization provides more technical details. For deployment information, we encourage you to visit our Technet site.

Windows Server “8” Beta: Hyper-V & Scale-up Virtual Machines Part 2…

2012-04-06T16:56:00Z

Today’s blog concludes the discussion of Hyper-V & Scale Up Virtual machines.

Jeff Woolsey, a Principal Program Manager on the Windows Server team, wrote this blog.

Virtualization Nation,

In the last blog, we discussed how Windows Server “8” introduces NUMA for virtual machines and how Hyper-V automatically does the right thing when creating a virtual machine. Let’s take this a step further and discuss what improvements have been made from the Windows 8 Developer Preview to the Windows Server “8” Beta (and Client Hyper-V in Windows 8 Consumer Preview).

Learning from the Developer Preview
Here’s what the GUI looked like in the Windows Server “8” Developer Preview.

Figure 2: Windows 8 Developer Preview Hyper-V VM NUMA Settings

During development, we found you were naturally interested in these NUMA settings and started reconfiguring them. (As expected …) However, once you were done experimenting with these settings, you weren’t clear how to correct the situation. What was needed was something like this:

Figure 3: Reset Button

And that’s exactly what we did. If you take a look at Hyper-V in Windows Server “8” Beta, the configuration has been redesigned. In addition, you’ll see there’s a new button (highlighted in green) labeled Use Hardware Topology. When you click this button, Hyper-V resets the virtual NUMA topology to the topology of the physical hardware.

Figure 4: Windows 8 Beta Hyper-V VM NUMA Settings

So, when in doubt, click the Use Hardware Topology button to get your optimal settings. Finally, I should mention that the virtual machine must be turned off before you can make any NUMA configuration changes or click the Use Hardware Topology button. I don’t know of any operating system that can handle having its NUMA configuration changed on the fly.

One additional benefit of Hyper-V virtual NUMA is that it helps future proof your investment. Whether you’re deploying existing solutions on this version of Hyper-V or architecting new solutions for the future, NUMA is prevalent today and its use is only increasing. In fact here’s another great example with Windows Server “8”: IIS 8.0.

Windows Server “8,” Hyper-V, IIS 8.0 & NUMA
For the first time, IIS is now NUMA aware. From http://learn.iis.net/page.aspx/1095/iis-80-multicore-scaling-on-numa-hardware/

IIS 8.0 addresses this problem by intelligently distributing and affinitizing its processes on Non-Uniform-Memory-Access (NUMA) hardware.

Internet Information Services (IIS) on Windows Server 8 is NUMA-aware and provides the optimal configuration for the IT administrators. Following section describes the different configuration options to achieve the best performance with IIS 8.0 on NUMA hardware.
IIS supports following two ways of partitioning the workload:

1. Run multiple worker processes in one application pool (i.e. web garden).
If you are using this mode, by default, the application pool is configured to run one worker process. For maximum performance, you should consider running the same number of worker processes as there are NUMA nodes, so that there is 1:1 affinity between the worker processes and NUMA nodes. This can be done by setting "Maximum Worker Processes" AppPool setting to 0. In this setting, IIS determines how many NUMA nodes are available on the hardware and starts the same number of worker processes.
2. Run multiple applications pools in single workload/site.
In this configuration, the workload/site is divided into multiple application pools. For example, the site may contain several applications that are configured to run in separate application pools. Effectively, this configuration results in running multiple IIS worker processes for the workload/site and IIS intelligently distributes and affinitizes the processes for maximum performance.

In addition, there are two different ways for IIS 8.0 to identify the most optimal NUMA node when the IIS worker process is about to start.

1. Most Available Memory (default)
The idea behind this approach is that the NUMA node with the most available memory is the one that is best suited to take on the additional IIS worker process that is about to start. IIS has the knowledge of the memory consumption by each NUMA node and uses this information to "load balance" the IIS worker processes.
2. Windows
IIS also has the option to let Windows OS make this decision. Windows OS uses round-robin.

Finally, there are two different ways to affinitize the threads from an IIS worker process to a NUMA node.

1. Soft Affinity (default)
With soft affinity, if other NUMA nodes have the cycles, the threads from an IIS worker process may get scheduled to non-affinitized NUMA node. This approach helps to maximize all available resources on the system as whole.
2. Hard Affinity
With hard affinity, regardless of what the load may be on other NUMA nodes on the system, all threads from an IIS worker process are affinitized to the chosen NUMA node that was selected using the design above.

Performance Monitoring & Virtual NUMA
You may be wondering, “How can I verify that the virtual machines I’m running are running using local CPU and memory resources as opposed to remote?” Hyper-V has that too. Take a look in Perfmon and you’ll notice two new counters:

1. Highlighted in green: For virtual processors there’s the Hyper-V Hypervisor Virtual Processor with a Remote Run Time counter.
2. Highlighted in red: Under Hyper-V VM VID Partition you’ll see the new Remote Physical Pages counter.

Figure 2: New Virtual NUMA Performance Counters

The LOWER the number (zero is ideal) the better. In this case, both numbers are zero (best case) meaning that all virtual processor and memory allocations are local.

Summary
With Windows Server “8” we want to help you cloud optimize your business and give you the ability to host a greater percentage of workloads on Hyper-V. In addition, we want to future proof your investments, which is why this version of Hyper-V :
• Supports massive scale-up virtual machines
• Introduces virtual NUMA and optimally configures virtual machine topology automatically

Finally, since you may not have a large scale-up system close at hand, I thought I’d leave you with a few screenshots. Cheers, -Jeff

Figure 3: Windows Server 2008 R2 SP1 as a Guest with 32 Virtual Processors

Figure 4: Windows Server “8” Beta as a Guest with 32 Virtual Processors

Figure 5: Centos 6.2 as a Guest with 32 Virtual Processors

Windows Server “8” Beta: Hyper-V & Scale-up Virtual Machines Part 1…

2012-04-05T17:04:00Z

The thing I love most about Microsoft is our focus on customers. Prior to coming to Microsoft, I spent the majority of my career working for companies that competed against Microsoft. Microsoft would release a new product into a market and sometimes they would nail it immediately and other times they didn’t. We brushed aside its strengths, would delight in pointing out any flaws, and congratulated ourselves on how much better we were. But we’d still make plans to sell our stock options before Microsoft got V3 out the door. I underestimated just how good Microsoft was at listening to customers and how courageous they were in doing whatever it took to satisfy those needs. Hyper-V has been a great feature and customers love it but we had some catching up to do. Well, we’ve been listening to customers and making some changes. Windows Server “8” is our V3 for Hyper-V and I think you are going to like it.

The Hyper-V V3 list of features is large and comprehensive so we’ll be discussing them through lots of team blog posts. Today we’ll start a two part blog focused on just one aspect of Hyper-V which is its support for scale up virtual machines. The raw numbers are impressive (32 logical processors; 1TB RAM; 64TB virtual disks) but as Jeff Woolsey shows in this post, increasing performance requires more than just increasing those numbers. Jeff describes our virtual NUMA (Non-Uniform Memory Access) support in Hyper-V V3. NUMA used to be an exotic technology for super high end servers but it is now used in almost all servers so it is important to support this well. Things that seem exotic and high end today are common and critical tomorrow. A number of workloads like SQL Server take advantage of NUMA to increase scalability and performance.

You might look at our V3 scale up numbers and NUMA support and think that they are well beyond anything you need - today. What that really means is that V3 gives you the freedom to stop worrying about which workloads you can virtualize and which ones you can’t. Hyper-V V3 provides the confidence that you can virtualize them all and that they will just work even when they have new resource requirements in the future. I encourage you to download the beta and see for yourself.

Jeff Woolsey, a Principal Program Manager on the Windows Server team, wrote this blog.

Virtualization Nation,

First, a huge thank you from everyone in Windows Server.

Since the release of Windows Server “8” Beta a few weeks ago, the feedback has been pouring in from all over the world and is overwhelmingly positive across all technologies in Windows Server. We’re both thrilled and humbled by your reaction.

Thank You.

Let’s Talk Scale Up
In the development of Windows Server “8,” one overarching goal was to create the Best Platform for Cloud. Whether that deployment is a in a small, medium, enterprise or massive hosted cloud infrastructure, we want to help you cloud optimize your business. One aspect to building the best platform for the cloud is the ability to host a greater percentage of workloads on Windows Server Hyper-V. Today, with Hyper-V in Windows Server 2008 R2, we can easily support virtualizing the majority of your workloads, but at the same time, we also recognize you want to virtualize your larger, scale-up workloads that may require dozens of cores, hundreds of gigabytes of memory, are likely SAN attached and with high network I/O requirements.

These are the workloads we targeted with Hyper-V in Windows Server “8”.

Scale Isn’t Just More Virtual Processors
One common Hyper-V question I hear is, “When will Hyper-V support more than 4 virtual processors? I need this for my scale up workloads.” While this is an understandable question, it’s also a surface level question that doesn’t look at the problem in its entirety. Let’s step back from virtualization and discuss a scenario with a physical server.

Suppose you deploy a brand new physical server with 8 sockets, 10 cores per socket with symmetric multithreading (SMT) for a total of 160 logical processors (8 x 10 x 2), but the server is only populated with 1 gigabyte of memory. With only 1 GB of memory, the number of logical processors is irrelevant because there isn’t enough memory to keep the compute resources busy. So, you add 512 GB of memory to help address the memory bottleneck.

However, there are still only two onboard 1Gb/E network interface cards (NICs) so you’re still going to be network I/O limited. Now, you add four 10Gb/E NICs which helps address the network I/O bottleneck. However, you still only have the system populated with a couple of hard disks so there’s your next bottleneck…

See my point?

The key to good performance and scale is balance. You have to look at the entire system holistically. You have to look at compute, memory, network and storage I/O, and address them all. Adding virtual processors doesn’t necessarily equate to good overall performance or scale.

As we investigated creating large virtual machines, we looked at the overall requirements to providing excellent, scale-up virtual machines. Today, we’re going to focus on the important relationship between CPU and memory. This means we need to discuss non-uniform memory access (NUMA). Before we discuss Hyper-V and scale-up virtual machines, let’s start with NUMA on a physical server.

Non-Uniform Memory Access (NUMA)
What is NUMA? NUMA was designed to address the scalability limits of a traditional symmetric multiprocessing (SMP) architecture. With traditional SMP, all memory accesses are posted to the same shared memory bus and each processor has equal access to memory and I/O. (BTW, about 10 years ago NUMA systems weren’t common, but with the rise of multi-core, multi-processor systems, NUMA is the norm!) Traditional SMP works fine for a relatively small number of CPUs, but quickly becomes an issue when you have more than a few compute resources competing for access to the shared memory bus . It only gets worse when you have dozens or even hundreds of threads…

NUMA was designed to alleviate these bottlenecks by grouping compute resources and memory into nodes. Each node is then connected through a cache-coherent bus. Memory located in the same NUMA node as the CPU currently running the process is referred to as local memory, while any memory that does not belong to the node on which the process is currently running is considered remote. Finally, this is called non-uniform because memory access is faster when a processor accesses its own local memory instead of remote memory. Here’s a simple example of a four NUMA node system.

Figure 1: Optimal NUMA Configuration

Figure 1 shows an optimal NUMA configuration. Here’s why:
1. The system is balanced. The same amount of memory is populated in each NUMA node.
2. CPU and memory allocations are occurring within the same NUMA node.

Now let’s contrast this with a non-optimal NUMA configuration.

Figure 2: Non-Optimal NUMA Configuration

Figure 2 shows a non-optimal NUMA configuration, and here’s why it’s non-optimal:
     1. The system is imbalanced. Each NUMA node has a different number of DIMMs.
     2. NUMA Node 2 has an odd number of DIMMs which means that if the system has the ability to use memory interleaving it may be unable to do so because the DIMMs are not installed in pairs. (This also depends on motherboard interleaving requirements: pairs, trios, etc.)
     3. NUMA Nodes 2 and 3 don’t have enough local memory, meaning some memory accesses are local to the node while some memory accesses are remote. This leads to inconsistent, non-linear performance.
     4. NUMA Node 4 has no local memory, so all memory accesses are remote. This is the worst possible scenario.

How SQL Server Uses NUMA
As you can see in the examples above, as more compute and memory resources are added, it’s a two-edged sword. On the one hand, there are more resources available for the workload to employ, but the operating system and application must manage additional complexity to achieve consistent scale and improved performance. Because of these unique requirements, scale-up workloads are developed to be “NUMA aware.” One great example is Microsoft SQL Server. Starting with SQL Server 2005, SQL Server is NUMA aware and uses NUMA to its advantage. This MSDN article provides insight:

SQL Server groups schedulers to map to the grouping of CPUs, based on the hardware NUMA boundary exposed by Windows. For example, a 16-way box may have 4 NUMA nodes, each node having 4 CPUs. This allows for a greater memory locality for that group of schedulers when tasks are processed on the node. With SQL Server you can further subdivide CPUs associated with a hardware NUMA node into multiple CPU nodes. This is known as soft-NUMA. Typically, you would subdivide CPUs to partition the work across CPU nodes. For more information about soft-NUMA, see Understanding Non-uniform Memory Access.

When a thread running on a specific hardware NUMA node allocates memory, the memory manager of SQL Server tries to allocate memory from the memory associated with the NUMA node for locality of reference. Similarly, buffer pool pages are distributed across hardware NUMA nodes. It is more efficient for a thread to access memory from a buffer page that is allocated on the local memory than to access it from foreign memory. For more information, see Growing and Shrinking the Buffer Pool Under NUMA.

Each NUMA node (hardware NUMA or soft-NUMA) has an associated I/O completion port that is used to handle network I/O. This helps distribute the network I/O handling across multiple ports. When a client connection is made to SQL Server, it is bound to one of the nodes. All batch requests from this client will be processed on that node. Each time the instance of SQL Server is started in a NUMA environment, the SQL error log contains informational messages describing the NUMA configuration.

Traversing NUMA Nodes and Remote Memory
You may be wondering, “How much is performance impacted by traversing NUMA nodes?” There’s no one answer because it depends on a variety of factors. Here are a few:
     1.  Is the system balanced? No type of software magic can save you if the system is configured poorly and imbalanced in the first place. (I put this at the top of the list for a reason…)
     2.  Is the workload NUMA aware? For example, SQL Server is NUMA aware, which means it looks at the underlying NUMA topology and tries to perform CPU and memory allocations with best physical locality.
     3.  Is the physical server a newer server with an integrated memory controller in the processor, or is this an older physical server with a front side bus (FSB)? If it’s the latter, there’s a much greater performance penalty if you have to traverse the front side bus for NUMA hops…

The important thing to remember is that to build an optimal scale-up system, you want to ensure that CPU and memory allocations are made with optimal physical locality. The best way to do this is by building a balanced system. There’s a quick primer on NUMA on a physical machine. Now let’s add Hyper-V to the mix. First, let’s briefly discuss Hyper-V before this release.

NUMA, Windows Server 2008 R2 SP1 Hyper-V & Earlier
In releases prior to Windows Server “8,” Hyper-V is NUMA aware from the host perspective. What this means is that the hypervisor allocates memory and CPU resources with best physical locality. By default, Hyper-V resource placement is based on the model that generally our customers’ workloads are bounded more by memory performance than by raw compute power. Placing a virtual machine entirely on one NUMA node allows that virtual machine’s memory to be 100% local (assuming it can be allocated as such), which achieves the best overall memory performance.

Here’s one simple example. In this example, let’s assume we have an active 4 virtual processor virtual machine with an average CPU utilization above 75%. With Windows Server 2008 R2 or Windows Server 2008 R2 with SP1, if we created a virtual machine with 4 virtual processors, Hyper-V would do this:

Figure 1: 4 Virtual Processor Virtual Machine on Windows Server 2008/2008 R2

As you can see in figure 1, Hyper-V creates a virtual machine and allocates resources optimally within a NUMA node. All CPU and memory allocations are local. Compare that with this:

Figure 2: A Non-Optimal Scheduling Example (Hyper-V Doesn’t Do This...)

As you can see in Figure 2, there’s a lot of NUMA node traversal.

From a virtual machine perspective, Hyper-V doesn’t present a NUMA topology within the virtual machine. The topology within the virtual machine appears as a single NUMA node with all local memory, regardless of the physical topology. In reality, the lack of a NUMA topology hasn’t been an issue with Hyper-V in Windows Server 2008 R2 SP1 and earlier because the maximum number of virtual processors you can create within a virtual machine is 4 virtual processors and the maximum amount of memory you can assign a virtual machine is 64 GB. Both of these fit into existing physical NUMA nodes, so providing NUMA to a virtual machine hasn’t really been an issue to date.

Scalability Tenet
Before we dive into what’s changing with this version of Hyper-V, let me first state an overarching scalability tenet:

>> Increasing the number of cores should result in increased performance. <<

I’m sure this sounds obvious and it’s certainly what you expect when you buy a system with more cores; however, there comes a point where increasing the number of cores may plateau or result in performance degradation because the cost of memory synchronization outweighs the benefits of additional cores. Here are two examples (sample data only) to illustrate my point.

Suppose you deployed a workload on a 32-way system ($$$$) only to find out that:
• In example A: workload performance plateaus at 8 processors? Or worse,
• In example B: workload performance peaks at 8 processors and degrades beyond that?

How would you feel about your return on investment on this large scale-up system?

Me too.

What you expect is as close to linear scaling as possible, which looks like this:

As we looked at scaling up virtual machines for this version of Hyper-V and creating virtual machines with 12, 16, 24, and up to 32 virtual processors per virtual machine, we knew this had to change. As the number of virtual processors per virtual machines exceeded the number of physical processors in a NUMA node, host side NUMA alone wouldn’t allow us to maximize hardware utilization. Most importantly, we wanted to ensure that guest workloads would be presented with the optimal information to work most efficiently and in conjunction with Hyper-V to provide the best scalability. To do that, we needed to enable NUMA within the virtual machine.

Windows 8 Hyper-V: Virtual Machine NUMA
With this version of Hyper-V, we’re introducing NUMA for virtual machines. Virtual processors and guest memory are grouped into virtual NUMA nodes and the virtual machine presents a topology to the guest operating system based on the underlying physical topology of compute and memory resources. Hyper-V uses the ACPI Static Resource Affinity Table (SRAT) as the mechanism to present topology information for all the processors and memory describing the physical locations of the processors and memory in the system. (Side Note: Using the ACPI SRAT for presenting NUMA topology is an industry standard, which means Linux and other operating systems that are NUMA aware can take advantage of Hyper-V virtual NUMA.)

With virtual NUMA, the guest workloads can use their knowledge of NUMA and self-optimize based on this data. It means Hyper-V works in concert with the guest operating system to create the best, most optimal mapping between virtual and physical resources, which in turn means that applications can ensure the most efficient execution, best performance and most linear scale. In addition, Hyper-V also includes fine grained configuration controls for virtual NUMA topologies which must be migrated across systems with dissimilar physical NUMA topologies. To provide portability across various disparate NUMA topologies, one needs to ensure that the virtual topology can be mapped among the physical topologies of all machines to which the virtual machine might be migrated.

At a high level, with two virtual machines under moderate to high load (75%+), it looks like this:

As you can see:
• Virtual machine 1 NUMA nodes A & B correspond with physical NUMA nodes 1 and 2
• Virtual machine 2 NUMA nodes A & B correspond with physical NUMA nodes 3 and 4

Let’s take a deeper look.

Virtual NUMA Example
In this example, the physical system is configured as: 1 socket with 8 cores and 16 GB of memory. When I create a new virtual machine, Hyper-V looks at the underlying topology and creates a virtual machine configured with NUMA as follows:
     • Maximum number of processors per NUMA node = 8
     •   Maximum amount of memory per NUMA node= 13730 (this number is calculated using the total and subtracting a reserve for the root)
     •   Maximum number of NUMA nodes allowed on a socket = 1

Here’s what the configuration looks like:

This is the optimal virtual NUMA topology for this physical system. At this point, you may be thinking, does this mean that we’ve added a whole bunch of complexity? Do I need to start counting cores, DIMM sockets, in all my servers?

No. Here’s a very important point:

>> By default, Hyper-V automatically does the right thing. <<

When a virtual machine is created, Hyper-V looks at the underlying physical topology and automatically configures the virtual NUMA topology based on a variety of factors including:
• The number of logical processors per physical socket (a logical processor equals a core or thread)
• The amount of memory per node

In addition, we’ve also provided advanced configuration options in those infrequent cases where it’s needed. We’ve included these advanced settings in case you’re moving virtual machines between hardware with disparate NUMA topologies. While this isn’t likely to happen often, we wanted to provide our customers the advanced capabilities to give them the flexibility they require. In the next blog post, we’ll discuss changes made between the Windows Server 8 Developer Preview and the Windows Server “8” Beta based on your feedback.

Standards-based Management in Windows Server “8”

2012-03-30T00:15:00Z

Microsoft Windows has long supported standards-based management. We were one of the founding members of the Distributed Management Task Force (DMTF) and shipped the first, and richest, Common Information Model (CIM) Object Manager (CIMOM) we all know as Windows Management Instrumentation (WMI). While WMI has served our customers and partners well, the true promise of standards-based management has not yet been realized. By and large, it is still a world where you have vendor-specific tools – Windows managing Windows, Linux managing Linux, and network & storage vendors managing their own stuff. Customers still have islands of management. There are examples of products which bridge these worlds but often they require bogging down the managed resource with extra vendor-specific agents. Lack of standards-based management is a major pain point for customers.

We spent a lot of time talking to partners and customers to understand what they needed to succeed with Windows Server “8”. We paid particular attention to Cloud OS scenarios and from there, it was clear that we needed a major investment in standards-based management. The shift to a Cloud OS focus significantly increased the scope of the management problem. Not only did we shift our focus from managing a server to managing lots of servers, but we also need to manage all the heterogeneous devices necessary to bring those servers together into a comprehensive and coherent computing platform. Today, cloud computing works by picking and qualifying a very small set of components and having a large staff of developers write component-specific management tools. Generalized cloud computing requires standards-based management. This is why we made a major investment in standards-based management in Windows Server “8”.

The heart of the management problem is that it requires a distributed group of people, often with conflicting interests, to make a common set of decisions. Our approach to this is simple: create a value-chain that makes it easy and rational to do the right thing. Development organizations look at how much it costs to develop something and what benefits it brings them. If the ratios work out, then they do the work, otherwise they don’t. So our job is to minimize the cost and effort to implement standards-based management and to maximize the benefit. This blog post describes how we accomplished that. It does not discuss our other major standards-based management initiative: Storage Management Initiative Specification (SMI-S) which allows Windows Server “8” to discover and manage external storage arrays. We’ll discuss that in a future blog post.

This blog post contains content appropriate for both IT Pros and developers. It contains both code and a schema example to highlight how easy we made things for developers. If you are an IT Pro, you might find this valuable in making good decisions for your deployment and architectural decisions in your IT infrastructure.

Wojtek Kozaczynski, a Principal Architect in the Windows Server team, wrote this blog.

--Cheers! Jeffrey

Background

WMI first shipped in Windows 2000 (and was available down-level to NT4). It used a COM-based development model and writing class providers was not for the faint of heart. Frankly, it was difficult to write them and more difficult to write them correctly. In addition to a difficult programming model teams had to also learn the new world of standards-based management with CIM schemas, the Managed Object Format (MOF) language, and other new terms, mechanisms and tools. We got quite good coverage in the first few releases but many teams were not satisfied with the effort-to-benefit ratio.

A big part of that equation is the benefit side. Starting a management ecosystem from scratch is incredibly difficult. If you write a provider and no one calls it, what value was generated? None. This is why Systems Management Server (SMS), now known as the System Center Configuration Manager (SCCM), added support for WMI around the same time we released it (WMI was actually spawned out of the SMS team.) This was great, but it had two problems:

It created an incentive to produce WMI providers which were largely focused on inventory and monitoring of the managed resources (vs. command and control), and
SMS was not widely deployed so the teams that wrote WMI providers didn’t see the customer impact of their investments.

Since the release of the WMI there has been a steady increase in the number of management products and tools that consume its providers, but for a long time this had not been matched by a proportional increase in coverage. Then things started to change with Hyper-V. The original management schemas defined by the DMTF were focused on what existed in the world, as opposed to focusing on management scenarios. There are things to be said for both approaches but when it came to managing virtual environments the DMTF got a team of pragmatists involved and when that schema came out, it was quickly adopted. The Hyper-V team developed providers that implemented the schema classes and the System Center Virtual Machine Manager (SCVMM) team produced a management tool which consumed them. This worked really well and it turned some heads because it demonstrated that the WMI was good for more than just inventory and monitoring. WMI could be effectively support configuration, command and control as well. To be fair, a number of other teams had done similar things before, but none of them had the visibility or impact that Hyper-V and SCVMM had.

The other big change in standards-based management was the definition and availability of a standard management protocol. WMI was a standard CIMOM that hosted many standard class providers, but at the time there wasn’t an interoperable management protocol, so WMI used DCOM. This, however, made it an island of management for Windows managing Windows. It worked well, but it did not deliver on the vision of standards-based management. That changed with the DMTF’s definition and approval of the WS-Management (WS-MAN) protocol, a SOAP-based, firewall-friendly protocol that allows a client on any OS to invoke operations on a CIMOM running on any platform. Microsoft shipped the first partial implementation of WS-MAN in Windows Server 2003/R2 and named it Windows Remote Management (WinRM). It interoperated with a number of CIMOM/WS-MAN stacks available on other platforms including Openwsman (Perl, Python, Java and Ruby Bindings), Wiseman (a Java implementation), and OpenPegasus.

Once standards-based management clients and CIMOMs could interoperate, the ball started rolling. However it also started stressing the seams in the increasingly heterogeneous world as vendors used the protocols to develop truly agentless management solutions. Differences in the ways the specifications got implemented meant that the tools needed to write special case code. Difficult APIs made it hard to write serious applications. Gaps in coverage meant that vendors still had to install agents, and vendors and customers hate having extra agents on the machines. Vendors hate them because they require lot of work to write and keep up to date with OS releases. Customers hate them because they complicate provisioning processes and introduce yet another thing that consume precious resources and can go wrong.

Why change?

Early in the Windows Server “8” planning process we realized that we could not deliver a Cloud OS without a major investment in standards-based management. There are simply too many things to manage in the cloud to manage each of them differently. Considering the situation I described above, we concluded that we needed to:

Dramatically reduce the effort required to write WMI providers and standards-based management tools
Substantially improve manageability coverage particularly in the areas of configuration, command and control
Update our code to comply with the latest DMTF standards
Tightly integrate WMI and Windows PowerShell
Provide a clear and compelling value proposition for everyone to use standards based management on Windows or any other platform.

Summary of what we have done

Let’s take a look at what we’ve done from two perspectives; the IT Pro perspective, and the Windows/ device developer perspective.

Our goal for IT Pros is to let them manage everything using Windows PowerShell, so we needed to give them simple-to-use cmdlets to remotely manage resources with standard interfaces on remote machines or heterogeneous devices. This, in turn, allows the IT Pros to script against those resources and write workflows which tie together tens, or tens of thousands of servers and devices without having to learn, configure and operate separate technologies and toolsets for each resource type.

Our goal for Windows/device developers is to make it simple and easy to define and implement standard-based management interfaces, and then expose them through client APIs, cmdlets and REST endpoints. For developers writing management tools we wanted to make it simple and easy to manage all the components of a solution including down-level DCOM Windows servers, standards-based management Operating Systems, servers and devices. For Web Developers we want to make it simple and easy to manage Windows via REST APIs.

Let’s start looking at what we have done starting from the developer’s perspective. The picture below shows the components of what we call the CIM stack.

In the area of provider development we introduced a new Management Infrastructure (MI) API for the WMI, which significantly simplifies development of new providers (MI Providers in the picture). New tools generate skeleton providers from the CIM class specifications. The new API supports rich cmdlets semantics that IT Pros have come to expect: –WhatIf, -Confirm, -Verbose as well as progress bars and the other cmdlet behaviors. When a new provider that supports the rich semantics is called by an old CIM client, these APIs do nothing. However new clients and Windows PowerShell can request these semantics and the APIs “light up” to deliver rich experience.
We made WS-MAN the primary protocol for management of Windows Servers and kept the COM and DCOM stacks for backwards compatibility. We have completed the full set of WS-MAN protocol operations and optimize our implementation for performance and scale. We also added support for handling connection interruptions to make management more robust. This simplifies the task of managing large sets of machines where interruptions are sure to occur.
For the client developers we created a new MI Client API and stack that can communicate with the WMI over COM locally and DCOM and WS-Man remotely. It can also communicate with any CIM-compliant server via WS-MAN. The client’s API, both C/C++ and .Net, is consistent with the provider MI API (they share the main header file).

The above gave us the foundation on which we have built Windows PowerShell access to CIM classes implemented in any CIMOM, which is illustrated in the picture below.

We created a Windows PowerShell module called CIM Cmdlets with tasks that directly correspond to the generic CIM operations. The module is built on top of the client .Net API and can manage any standards-based management resource.
We modified Windows PowerShell to be able to generate resource specific CIM-Based cmdlets at run-time. These cmdlets are generated from a declarative XML specification (CDXML) of the Windows PowerShell-to-CIM mapping and can call CIM class providers locally or remotely. This allows a developer to write a CIM provider, write the CDXML mapping file, and make the generated cmdlets available on every Windows device running Windows PowerShell 3.0. This works for non-Windows providers as well. Now imagine the value of this to device vendors. If they implement a standards-compliant provider and include this CDXML mapping file, then a couple hundred million Windows machines will be able to manage that device without the vendor having to write, debug, distribute or support any additional code. When a new version of Windows comes out, their device is supported without them having to write any code. This alone gives a huge incentive to the device vendors to support standards-based management.

In the picture above you may have noticed a box labeled “NanoWBEM”. Let’s talk about that now. As we engaged our partners and the community in our plans to pursue standard- based management we got mixed reactions. Some felt it was the right thing to do and understood the business opportunities it could create, but where skeptical about whether it would really work. When we drilled into that we discovered that the partners did not feel like they could succeed using the existing open source CIMOMs. At the same time our own System Center team encountered similar problems as it expanded its capabilities to manage Linux Servers. To address them the team started a project to build a portable, small-footprint, high performance CIMOM and the result is the NanoWBEM. NanoWBEM is written in portable C and runs on Linux, UNIX and Windows. Because of its very small size it is suitable for running on small devices such as network devices, storage controllers and phones. NanoWBEM uses the same MI provider APIs as WMI, so the same tools that the developers use to create Windows providers can be used to develop providers for other platforms.

Now to address the original concerns of our partners and the community will are planning to make NanoWBEM available to the open-source community.

With the things I described above we have the best of both worlds:

We give IT Pros powerful tools to access the standard-based management APIs realized by CIM class providers. If those classes are implemented by the MI providers, they can support the extended Windows PowerShell semantics like progress, -WhatIf, -Confirm and –Verbose.
We also gave the managed software and device developers tools to create new MI providers at a significantly lower cost than before, and make them manageable by IT Pros via Windows PowerShell modules at a very small incremental cost.

Finally, for the Web developers that want to manage Windows from non-Windows platforms we have developed the Management OData IIS Extension. This contains tools and components that simplify building REST APIs (OData Service endpoints).

OData is a set of URI conventions, tools, components and libraries for building REST APIs. What makes the OData services stand out is that they are based on explicit domain models, which define their data content and behavior. This allows rich client libraries (e.g. Windows/IoS/Android Phones, Browsers, Python, Java, etc.) to be generated automatically to simplify the developing solutions on a wide range of devices and platforms.

There are a number of products that have full Windows PowerShell APIs and need REST APIs now that they are being hosted in the Cloud. This is why our first use of OData focused on exposing set of Windows PowerShell cmdlets.

However we have architected a general purpose solution to for future releases. Rest APIs, and OData in particular, map very well to CIM data models so what we did was to provide a mechanism to map sets of cmdlets into a CIM data model and then expose that data model as an OData service endpoint.

A shallow dive into the CIM stack

In the preceding section, I showed a high-level overview of what we have done. This inevitably left many of you asking; So how does it work in practice? In the team blog we will take deep-dives into all features and components of the Windows Sever “8” management platform. In the meantime, for the inpatient among us, I will do a “shallow dive” into the features starting with the IT Pro experience.

CIM cmdlets

IT Pros have two mechanisms to manage CIM classes. The first option is to use the generic CIM cmdlets from the CimCmdlets module, which is imported to PowerShell_ISE and PowerShell by default. The cmdlets of the module should look quite familiar to IT Pros familiar with CIM because they map very directly to the generic CIM/WS-Man operations. For example three different parameter sets of the Get-CimInstance cmdlet map directly to CIM/WS-MAN generic operations: GetInstance, EnumerateInstances and QueryInstances. The module also includes cmdlets to create remote server connections (sessions) and inspecting definitions of classes registered with the CIMOM.

The new CIM cmdlets are a replacement for the *-Wmi* cmdlets which only worked in Windows-to-Windows. The cmdlets are optimized to work over WS-MAN and will continue to work seamlessly over DCOM, so as an IT pro you no longer need to use two sets of commands to manage Windows and Non-Windows machines.

The following example shows getting the names and types of the properties of the Win32_Service class registered in the WMI root\cimv2 namespace on the local computer.

Getting the names of the Win32 services from a remote server is as simple as this.

CIM-Based Cmdlets generated from CDXML

IT Pros can also use cmdlets that Windows PowerShell generates using a CDXML mapping file. This model allows developers to write one piece of code and get the benefits of both the Windows PowerShell and WMI ecosystems. It also allows cmdlets to be written in native code which was of particular interest to some of the OS feature teams. The CIM-based cmdlets, although written as WMI providers look and feel just like Windows PowerShell cmdlets:

They provide task-oriented abstractions that hide implementation details like namespace, class name, method names, etc.
They support full Windows PowerShell semantics: -WhatIf, -Confirm, etc…
They have uniform support for rich operational controls: -AsJob, -Throttlelimit, etc…
They are packaged as Windows PowerShell modules and are discoverable using the get/import-module cmdlets.

The CDXML file used to generate the CIM-based cmdlets maps a cmdlet verb, noun and parameters to a Cmdlet Adapter. A Cmdlet Adapter is a .NET class which maps the requirements of a Windows PowerShell cmdlet into a given technology. We ship a Cmdlet Adapter for CIM classes but anyone can write their own (e.g. to map cmdlets to a Java classes). The file extension of the mapping file is .CDXML (Cmdlet Definition XML). A number of related CDXML files can be combined into a Windows PowerShell module together with files which describe the returned objects and how to format them.

The beauty of this mechanism is that Windows PowerShell can import such a .CDXML module from a remote CIMOM, and then create cmdlets which manage the classes on that server without any prior knowledge about them. In other words a CIMOM can expose a server-specific Windows PowerShell interface to its classes at run-time, without a need for any software installation!

Authoring CDXML files requires a level of detail comparable with specifying any other cmdlet, plus information about mappings to the CIM class functions. To simplify that task we developed CDXML editing tools that we will detail in a separate blog. Without going into details let me illustrate the idea behind generated CIM-based cmdlets with a simple example. Above I showed how to access the Win32_Service class and its instances using the CIM cmdlets. Below is a .CDXML file that defines the Get-Win32Service generated CIM-based cmdlet that will call the enumeration method on the same class. You will not find the name of that Get-Win32Service cmdlet in the file because it is generated by default from the default noun Win32Service and the verb Get. What is in the file is the <QueryableProperties> element which defines the properties that Windows PowerShell will use to query for the instances of the Win32_Service class. In our case the property we want to query on is service Name.

The following sequence of Windows PowerShell commands imports the .CDXML file as a module, lists our new cmdlet as defined in the module, and then shows its signature. Notice that because in the .CDXML file we say that the parameter Name is mandatory (<cmdletParameterMetadata IsMandatory="true" cmdletParameterSets="ByName" />), Name it is shown as the only mandatory parameter in the Get-Win32Service cmdlet signature.

Note: If you are curious about how we accomplish this you can see the cmdlet we generate on the fly using the following command.

Our newly created cmdlet (Get-Win32Service) behaves like any other cmdlet and as I mentioned above, can be executed as a background job. Throttling (-ThrottleLimit) is useful when executing the command against a large set of servers. You can run the command against a few hundred or thousand servers and throttle how many concurrent outstanding requests are allowed to run.

We shipped Windows PowerShell V1 with 130 cmdlets and Windows PowerShell V2 with 230 cmdlets. With Windows PowerShell V3, Windows Server “8” ships with over 2,300 cmdlets. A large percentage of those cmdlets are written using the new WMI MI providers and .CDXML files. What that means is that those functions are both available via Windows PowerShell and via standards-based management. We recently shocked an audience by demonstrating the ability to install a set of Windows Server Roles from a Linux client machine using WS-MAN!

The CIM Client .Net API

Both CIM cmdlets and CIM-based cmdlets in Windows PowerShell are implemented on top of the new MI .Net Client API. Although it is unlikely that IT Pros will write C# client code, management tools developers certainly will, so let’s take a look at a simple example.

The client API supports both synchronous and asynchronous interactions with the server. The synchronous operations return IEnumerable<CimInstance> collections of CIM class instances. The asynchronous operations use the concept of asynchronous, observable collections from the Reactive Extensions, which results in efficient, simple and compact client code. Below is an example of a simple command-line program that enumerates instances of the Win32_Service class on a remote computer. My objective is not to discuss the client API, but to illustrate how compact and clean the resulting program is.

The code that handles the numeration is in the three highlighted lines of the Main function. The lines turn the result of enumerating instances of Win32_Service into an Observable collection of CimInstance objects and associate the consumer observer object with that collection. The observer contains three callbacks that handle returned instances, the final result, and errors. This makes it simple and easy to perform rich management functions against a remote CIM server in just a few lines.

The New WMI Providers

I said earlier that we significantly simplified development of the MI providers. There are a number of things that contributed to that simplification.

The picture below shows the steps involved in writing a provider. A provider can implement one or more CIM classes and the first step is to describe them in the MOF specification language. The next step is to generate a provider skeleton to implement the CIM classes. We provide a utility Convert-MofToProvider.exe to automate this step. This utility takes the class definition(s) as input and creates a number of C header and code files. Two of these files are worth mentioning

The first one, called the schema file, contains definitions of all data structures, including the CIM class instances, which the provider uses. It makes the provider strongly typed, and a pleasure to work with Visual Studio’s intellisense and auto-completion. This file should never be edited by hand.
The other file is the provider code file, which contains the skeleton of the provider. This is the only file that should be edited. It contains the code of all the CIM class methods, with the method bodies returning the not-implemented result. So the generated provider is buildable, can be registered and will run, but will do nothing.

The next step is to fill the CIM class methods with their respective logic. Once that is done, the provider can be registered and tested. We also greatly simplified the provider registration by building a new registration tool that takes only one input; the provider DLL. We could do that, because the MI providers have their class definitions complied into them in the schema file.

In order to make the new providers work well with Windows PowerShell we added the extended Windows PowerShell semantics API. The essence of that feature is that a provider can obtains input from the user while an operation is executing if the cmdlet that invoked the operation contains the –Confirm or –WhatIf parameter. The following code snippet is from a test provider that enumerates, stops and starts Win32 services, and illustrates how the feature works. The code is part of the operation that stops the service and asks the user (the MI_PrompUser() function) if she wants to stop the service with the name that was given to the operation as the Name argument. If the answer is No (bContinue == FALSE) or the function failed, which means the provider could not reach the user, the provider does not stop the process but writes a message back to the user (the MI_WriteVerbose() function) and terminates the operation.

Management OData

The last feature I want to briefly describe is the IIS extensions for building OData-based RESTful management endpoints. The idea behind the feature is that we can declaratively configure how to dispatch the endpoint service requests to cmdlets in a Windows PowerShell module. Let me explain how it works using an example of a very simple endpoint that can be used to enumerate, create and terminate Windows processes.

The endpoint’s directory includes the three files shown in the picture below.

The schema file contains the definitions of the endpoints’ entities and is loaded when the endpoint is loaded. The module file defines the Windows PowerShell cmdlets that will be called to handle the endpoint requests. The dispatching file ties the two other artifacts together by defining what cmdlets are called for different client requests. In our example it maps the query for Win32 processes onto the Get-Win32_Process cmdlet, which then uses a generic CIM cmdlet to talk to the WMI. The result of this endpoint configuration is shown in the screenshot below, which is a response to the URL

http://wojtek8srv3:8000/Management/Process.svc/Win32Process?$filter=Name eq 'svchost.exe'&$select=ProcessId.

In Summary

Jeffrey often makes the point that “nobody cares about your first million lines of code”. The “million” is an arbitrarily picked very large number, but the idea of the metaphor is that every software product must accumulate a critical mass of foundational components before delivering a meaningful value. However once that critical mass is reached, even a few additional lines can make a significant difference.

Following that metaphor, I feel that in Windows Server “8” we have written our “first million” lines of the standards-based management platform code. We bridged the gap between the IT Pros who are managing increasingly complex cloud infrastructures and the Windows and device developers who build the things that must be managed. We have laid a consistent foundation that spans from low level CIM interfaces on one end, to the IT Pro oriented Windows PowerShell and OData interfaces on the other end. We’ve created a clear and compelling value for heterogeneous devices to implement standards based managed and we’ve delivered comprehensive coverage so that management tool providers can use standards based management to manage Windows.

Microsoft Online Backup Service

2012-03-28T08:28:00Z

A number of people view the cloud in all or nothing terms – you move everything to the cloud or you leave everything where it is. The reality is that a lot of cloud adoption is going to follow the hybrid approach using both on-premises and cloud services. An example of this hybrid approach is Windows Server “8” which gets better when you connect it to the cloud. Today’s blog by Gaurav illustrates that by describing the Microsoft Online Backup Service, a new cloud based service for Windows Server “8”, which backs up your server’s data to the cloud so that it is safe and secure. That means that you no longer have to figure out where to store your backups so that they are safe. The cloud does that for you. This is an extensible model so it provides a great opportunity for partners to provide cloud backup solutions as well.

Gaurav Gupta, a Senior Program Manager on our Cloud Backup team, authored this blog

--Cheers! Jeffrey

Overview

The Microsoft Online Backup Service is a cloud based backup solution for Windows Server “8” which allows files and folders to be backed up and recovered from the cloud in order to provide off-site protection against data loss due to disasters. The service provides IT administrators with the option to back up and protect critical data in an easily recoverable way from any location with no upfront hardware cost or resources other than an Internet connection. This service is built on the Windows Azure platform and uses Windows Azure blob storage for storing customer data. Windows Server “8” uses the downloadable Microsoft Online Backup Agent to transfer file and folder data securely and efficiently to the Microsoft Online Backup Service. Once installed, the Microsoft Online Backup Agent exposes its functionality through the familiar Windows Server Backup interface.

Getting started

Getting started with Microsoft Online Backup service on Windows Server “8” Beta is a simple two-step process:

1. Get a free beta Online Backup Service account (with 10 GB of cloud storage). In order to request access, please sign up to get an invitation to the service at http://connect.microsoft.com/onlinebackup. Please note that there are a limited number of customers that we can support during the beta and we will grow our capacity over time. We have available slots right now so if you are willing to give us feedback, sign up now to try this great new service.
2. Download and install the Microsoft Online Backup Agent. The agent installer download is located on the Microsoft Connect Site indicated above.

Key features

Below are some of the key features we’re delivering in Windows Server “8” using Microsoft Online Backup service:

Simple configuration and management. Microsoft Online Backup Service integrates with the familiar Windows Server Backup utility in order to provide a seamless backup and recovery experience to a local disk, or to the cloud.

Simple user interface to configure and monitor the backups.
Integrated recovery experience to transparently recover files and folders from local disk or from cloud.
Easily recover any data that was backed up onto any server of your choice.
Windows PowerShell command-line interface scripting capability.

Figure 1: Microsoft Online Backup User Interface

Block level incremental backups. The Microsoft Online Backup Agent performs incremental backups by tracking file and block level changes and only transferring the changed blocks, hence reducing the storage and bandwidth utilization. Different point-in-time versions of the backups use storage efficiently by only storing the changes blocks between these versions.
Data compression, encryption and throttling. The Microsoft Online Backup Agent ensures that data is compressed and encrypted on the server before being sent to the Microsoft Online Backup Service over the network. As a result, the Microsoft Online Backup Service only stores encrypted data in the cloud storage. The encryption passphrase is not available to the Microsoft Online Backup Service, and as a result the data is never decrypted in the service. Also, users can setup throttling and configure how the Microsoft Online Backup service utilizes the network bandwidth when backing up or restoring information.
Data integrity is verified in the cloud. In addition to the secure backups, the backed up data is also automatically checked for integrity once the backup is done. As a result, any corruptions which may arise due to data transfer can be easily identified and they are fixed in next backup automatically.
Configurable retention policies for storing data in the cloud. The Microsoft Online Backup Service accepts and implements retention policies to recycle backups that exceed the desired retention range, thereby meeting business policies and managing backup costs.

The Microsoft Online Backup Service only supports the Windows Server “8” operating system. It does not support Windows 8 Consumer Preview client operating systems or any Windows operating systems released prior to Windows Server “8”.

Detailed steps

In this section, I am sharing some of the actions you would need to take for setting up your Windows Server “8” to backup or recover data from Microsoft Online Backup Service. To learn more technical concepts, functionality, and troubleshooting methods for Microsoft Online Backup Service, you can download the Understand and Troubleshoot Guide (UTG) from http://www.microsoft.com/download/en/details.aspx?id=29005

Registering to Microsoft Online Backup Service

You can choose the Register Server action in the Microsoft Online Backup MMC snap-in to start the registration wizard, and sign in using a pre-provisioned Microsoft Online Services ID. You also need to set the passphrase to encrypt the backups from the server.

Figure 2: Register Server Wizard Account Credentials page

Figure 3: Encryption Settings

Using Windows PowerShell:

The following code sample illustrates how you can use Windows PowerShell to register a server with Microsoft Online Backup Service after establishing variables to use to supply your credentials.

$pwd = ConvertTo-SecureString -String <password> -AsPlainText –Force

$cred = New-Object –TypeName System.Management.Automation.PsCredential –ArgumentList <username>, $pwd

Start-OBRegistration -Credential $cred

(As a side note, there is a very good blog post on how to import/export credentials into powershell scripts)

Once the server is registered you then need to use the Set-OBMachineSetting cmdlet to specify the encryption passphrase.

$pass = ConvertTo-SecureString -String <password> -AsPlainText –Force

Set-OBMachineSetting -EncryptionPassphrase $pass

Setup backup schedule

You can choose the Schedule Backup action to start the Backup Schedule Wizard to set the backup schedule in which you specify what files and folders you want to backup, how frequently you want to backup and how long you want to retain the backups in the cloud.

Figure 4: Schedule Backup Wizard: Item selection

Figure 5: Schedule Backup Wizard: Schedule time configuration

Figure 6: Schedule Backup Wizard: Retention Settings

Using Windows PowerShell:

To start a new backup from within Windows PowerShell, administrators need to define a backup policy, the data locations, the schedule for the backup job and the data retention policy for the backup policy. The following commands will setup variables for a basic backup job within Windows PowerShell.

$policy = New-OBPolicy

$filespec = New-OBFileSpec -FileSpec C:\Windows\Logs

$sched = New-OBSchedule -DaysofWeek Wednesday -TimesofDay 09:30

$ret = New-OBRetentionPolicy

The Windows PowerShell script will create a new Microsoft Online Backup Service policy named $policy that will back up all of the files and folders in the C:\Windows\Logs directory. The backup policy runs on Wednesdays at 9:30 AM and will keep the data in the backup for a period of 30 days.

To create the policy so that it runs the next time it encounters the scheduled task, do the following:

Add-OBFileSpec -Policy $policy -FileSpec $filespec

Set-OBSchedule -policy $policy -schedule $sched

Set-OBRetentionPolicy -policy $policy -retentionpolicy $ret

These commands put all of the previous variables into the $policy Microsoft Online Backup Service policy object so that it can be run at the next scheduled time.

If this is the first backup on the server after registration, the encryption passphrase must be set.

$passphrase = ConvertTo-SecureString <passphrase> -asplaintext -Force

Set-OBMachineSetting -EncryptionPassphrase $passphrase

Finally, save the Online Backup policy for it to be scheduled. Do this by running:

Set-OBPolicy -policy $policy

Start backup

After configuration of the backup schedule is completed, backups will occur as per the configured schedule. You can choose to invoke manual backup using the Backup Now actions in Microsoft Online Backup MMC snap-in. Backup Now runs the schedule immediately and the backups include only currently selected items. If you want to backup additional files or folders not included in the original schedule, you will need to modify the backup schedule before using Backup Now.

Figure 7: Backup Now option in Actions page

Using Windows PowerShell:

You can use the Start-OBBackup command to start the backup immediately using the value held in the backup policy:

Get-OBPolicy|Start-OBBackup

Recovering data

To recover data using the Microsoft Online Backup Service feature, click the Recover Data action in the Microsoft Online Backup MMC snap-in. As part of the Recover Data Wizard, you can choose the volume from where data needs to be restored and the date and time of the data to be restored. After selecting the date and time, you are presented an explorer view of the data which is backed up from where you can then select the items that you want to restore.

Figure 8: Recover Data Wizard: Select Volume and Date

Figure 9: Recover Data Wizard: Select Items to Recover

Using Windows PowerShell:

To recover data using Windows PowerShell, you can use the following commands to select the data which needs to be recovered:

$source = Get-OBRecoverableSource

$item = Get-OBRecoverableItem -Source $source[0]

$FinalItem = Get-OBRecoverableItem -ParentItem $item[0]

To run a recovery operation, use:

$recover_option = New-OBRecoveryOption

Start-OBRecovery -RecoverableItem $FinalItem -RecoveryOption $recover_option

Extensibility for partners

Partners who want to offer their own cloud backup solution by integrating with Windows Server Backup, please refer to our Cloud Backup Provider API Reference at http://msdn.microsoft.com/en-us/library/hh437322(v=vs.85).aspx

Conclusion

The Microsoft Online Backup Service gives IT administrators a powerful but simple option for off-site data protection of their critical file and folder data on Windows Server “8”.

Windows Server 2008 R2 Hyper-V has passed Common Criteria Evaluation Assurance Level 4+ (EAL 4+)

2012-03-22T23:23:00Z

I’m going to interrupt our flow of Windows Server “8” blog posts to share this hot off the press news regarding Windows Server 2008/R2 and Common Criteria.

David Cross, Partner Program Manager in Windows Server, authored this post.

--Cheers! Jeffrey

I am happy to announce that Windows Server 2008 R2 Hyper-V has passed the Common Criteria Evaluation Assurance Level 4+ (EAL 4+). Over the past 10 years, Microsoft has continued to demonstrate leadership in certifying our operating systems and applications, in order to provide customers with the additional confidence of independent validation of our design and security engineering.

Similar to our previous operating system evaluations and the independent Windows Server 2008 Hyper-V evaluation, we used the Windows Server 2008 R2 evaluation to demonstrate how Hyper-V is methodically designed, tested, and reviewed from a security perspective. The Hyper-V evaluation focused on isolation between the host partition and guest partitions, isolation between guest partitions and the Live Migration capability, which was added in Windows Server 2008 R2.

The Hyper-V evaluation was performed by atsec Information Security GmbH and certified by the Federal Office for Information Security (BSI), the body of the German government which certifies products according to the ITSEC criteria for evaluating computer security and the Common Criteria (CC).

Windows Server “8” will continue to build on this capability and security in Hyper-V and the Windows Server platform to provide consistent, reliable and secure platform solutions for private clouds and datacenters. We encourage customers to explore and build on the evaluated Hyper-V platform to meet your needs for the next generation applications and services!

David B. Cross
Windows Server

Building an Optimized Private Cloud using Windows Server “8” Server Core

2012-03-20T12:05:00Z

I like to say that Windows Server “8” is a transformational release. We stopped thinking of ourselves as an OS for an individual server and started thinking of ourselves as an OS equally suited for a single server or a cloud using lots of servers. We’ve always been concerned about our customers’ Capital/Operation expense (Capex/Opex) but when we think about customers with deployments of tens or thousands (or hundreds of thousands) of servers, everything changes. Efficiency, operational agility and automation features change from being nice-to-haves to being critical. Everyone knows these things but running our own cloud services really drives the point home. It’s the difference between knowing to keep your hands up because your boxing coach told you to and knowing it because you just got hit in the face. There is nothing like experience to motivate real and lasting change.

All the things we did for the cloud make things better for every customer, even those with a single server. If you choose not to use the automation we deliver, you’ll still benefit from all the great tools that our automation enables and, of course, everyone benefits from reduced patching and reboots.

Server Core is at the heart of our Cloud OS effort. We started investing in Server Core with Windows Server 2008. That wasn’t a huge hit, but we now think we’ve achieved critical mass with Windows Server “8” and are recommending it as the preferred configuration. We are telling our independent software vendor partners that they need to support Server Core and move to a remote GUI model. We still provide “Server with a GUI” as a compatibility option and added “Minimal Server Interface” as an option for those applications or admins that want some of the benefits of Server Core but need a little more time to make the full transition. This blog gives the background and some of the details behind our Server Core initiative.

David Cross, Partner Program Manager in Windows Server, authored this post.

--Cheers! Jeffrey

Building an Optimized Private Cloud using Windows Server “8” Server Core

Private clouds offer the promise to deliver scalable, dynamic, multitenant-aware services with minimal capital purchases and operational expenses. Private clouds deliver their true potential when deployed on an optimized platform running an optimized Server operating system. A cloud-optimized Server operating system delivers:

Multi-tenancy and scaled remote management
Consistent automation and scripting
Flexible settings and configuration
Low cost storage and reliability
Efficiency and power management
Minimized patching and external attack surface
Open web and application development model
Optimized size and footprint

These are the things we invested in for Windows Server “8” and are among the many reasons why it is the best cloud-optimized operating system. In planning the release, we spent over $10 million dollars and a year talking to the community and cloud solution builders. In those conversations, we heard the clear message that deployment agility and optimization were critical to your success. Let’s talk about what we have done to respond to that feedback.

As we highlighted and shared in our earlier blog post, the recommended deployment configuration for Windows Server “8” is Server Core. Server Core minimizes the disk space and memory requirements for Windows Server, enabling administrators to increase the density of their virtual machines and scale out their deployments significantly. We not only accomplish considerable space savings, reducing storage costs, but we also minimize the attack surface area, thereby also increasing security and reliability. Naturally, reducing the overall footprint of Windows Server also limits the number of components that must be patched on a given server. The minimum number of applicable patches reduces the frequency of reboots and increases server availability.

Looking at what we have advanced in the Windows Server “8” beta, you can see in the following table how we have focused on provided the capability, manageability and agility needed for deploying applications and services in a private cloud:

Feature	Windows Server 2008	Windows Server 2008 R2	Windows Server “8”
Roles/Product support	9 roles	10 roles	13 roles and SQL Server 2012 support
Manageability	Command line scripting only	PowerShell + 230 cmdlets	PowerShell + greater than 2,430 cmdlets and Server Manager support
Flexibility	Mutually exclusive: install in Server Core mode or in Full Server mode only	Mutually exclusive: install in Server Core mode or in Full Server mode only	Ability to add/remove roles to move between Server with a GUI Minimal Server Interface Server Core as well as a features on demand capability

Let’s now drill in a little more into our approach in this release; but first, let me provide a little history of Server Core to set some context for those not familiar with this deployment model. For many years, most of the Windows operating system was delivered as a monolithic component known as Windows Foundation. Windows Foundation included Windows Explorer, .NET framework, desktop shell, drivers, multimedia support, and Internet Explorer. Optional features and server roles were separate components that could be installed on top of the Windows Foundation.

In Windows Server 2008, we introduced Server Core with the goal of allowing you to only install those Windows components needed by your application. Server Core is a separate installation option and optional features and server roles are separate components which can be installed on it. This approach maximizes the server resources available to your application while reducing the security and serviceability footprint of this server. Server Core delivered a significant reduction in reboots due to reduced patching needs but only for a limited number of roles and workloads. Customer feedback showed that the adoption of Server Core in Windows Server 2008 was limited for the following reasons:

Customers needed roles and products (SQL Server in particular) that were not available on Server Core. Only 9 of 17 Server Roles ran on Server Core
Lack of PowerShell and no Server Manager support made it difficult to manage.
There was no conversion between Server Core and Full Server. If you installed Server Core and needed to switch to Full Server, you needed to start over.

Figure 1. Windows Server 2008 component structure.

Next Generation Server Core

While the first release of Server Core did not have wide adoption, it was very successful for those customers that used it. In Windows Server 2008 R2, we expanded Server Core support to several new server roles and added the .NET Framework and PowerShell. This increased the number of people that could use Server Core and made it easier for those that were using it. However, Server Core remained a separate Windows installation option with no path to a full installation. Based on customer feedback and desires to more widely use Server Core in private cloud deployments, we have made a large investment in Server Core capabilities in Windows Server “8” to increase deployment flexibility and also bring Server Core support to more server roles. The following roles are supported on Windows Server “8” in Server Core mode:

Active Directory Certificate Services
Active Directory Domain Services
Active Directory Lightweight Directory Services (AD LDS)
Active Directory Rights Management Server
DHCP Server
DNS Server
File and Storage Services (including File Server Resource Manager)
Hyper-V
Print and Document Services
Remote Desktop Services sub roles
- Remote Desktop Connection Broker
- Remote Desktop Licensing
- Remote Desktop Virtualization Host
Routing and Remote Access Server
Streaming Media Services (available as a separate download)
Web Server (including a subset of ASP.NET)
Windows Server Update Server

In addition, many server applications such as SQL Server 2012 will support Server Core in Windows Server “8.”

From an architectural perspective, we have streamlined the Server Core component to serve as a minimal common base for all Windows Server editions, and refactored the Windows Foundation into several components that can be installed and uninstalled individually. For example, with a single command, it is now possible to go from a Server Core machine with a command-prompt-only user interface to Server with a GUI with the complete Windows desktop. Moreover, it is just as easy to do the reverse transition. Either transition can be accomplished in only a few minutes and requires (at most) a single reboot.

Figure 2. Windows Server “8” component structure.

If you have the Windows Server “8” Beta installed, you can try these new capabilities yourself. To convert a Server with a GUI installation of Windows Server “8” to Server Core, start PowerShell and run the following command:

Uninstall-WindowsFeature Server-Gui-Mgmt-Infra -Restart

To re-install the GUI components, start PowerShell and run:

Install-WindowsFeature Server-Gui-Shell -Restart

One of our goals in Windows Server “8” was to maintain Server Core as streamlined as possible by allowing customers to limit their installs to “just enough” of Windows to fulfill their server’s desired function. For more information on how we did this see the Reducing the Disk Footprint with Features on Demand section below.

The Minimal Server Interface

Over the past two releases, service providers and IT professionals shared their needs for management and configuration of the operating system. In reality, many administrators love the agility and capability of the Server Core deployment mode, yet some want the flexibility of having the graphical tools available on the server without the overhead of the entire GUI and desktop.

The largest components (in terms of storage requirements) in Windows Server are comprised of the graphical user interface, or GUI. Based on historical patch data, GUI components are typically patched more frequently than their non-GUI counterparts. Of the GUI components in Windows, the two largest are Windows Explorer and Internet Explorer. Incidentally, these are also used rather infrequently on Windows Server installations. Accordingly, we moved Windows Explorer and Internet Explorer into an optional package, which can be enabled or disabled through the Server Manager interface, PowerShell, or the command line.

In Windows Server “8,” we are introducing a new experience called the Minimal Server Interface that enables most local GUI management tasks without requiring the full GUI Shell or Internet Explorer to be installed. The Minimal Server Interface contains binaries that provide comprehensive local management capabilities. To configure a Windows Server “8” machine with the Minimal Server Interface, ensure that the Graphical Management Tools and Infrastructure package is enabled and that the remaining Desktop Experience and Server Graphical Shell packages are disabled.

Figure 3. Feature selections showing the Minimal Server Interface

The Minimal Server Interface is actually a Server with a GUI install excluding Internet Explorer, Windows shell components such as the desktop, Windows Explorer, Metro-style application support, and the Desktop Experience. MMC and the new Server Manager, which can be used to manage local and remote Windows servers as well as serving as the new launch point for Windows server management tools, are both included in the Minimal Server Interface. Because the Minimal Server Interface does not include Explorer or the Server Graphical Shell, not all GUI management functionality is available. Namely, control panel applets implemented as shell extensions are not available. These include:

Programs and Features
Network and Sharing Center
Devices and Printers Center (however, Device Manager is available)
Display settings (however, there is a new tool called SetRes to allow the display resolution to be changed)
Firewall control panel (however, Advanced Firewall MMC snap-in is available)
Windows Update
Fonts
Storage Spaces

Most MMC snap-ins can also be installed independently of their corresponding roles using the Remote Server Administration Tools (RSAT) optional feature. In some cases, certain functionality within MMC snap-ins may be limited. For example, local help may not be available or the details pane of a snap-in may be constrained without Internet Explorer to display HTML.

The Minimal Server Interface, though it is not as large as an installation that includes the Server Graphical Shell, still requires roughly 4 GB more disk space than Server Core alone. Thus, we wanted to make sure that even the Minimal Server Interface can be removed whenever it is not needed. The Graphical Management Tools and Infrastructure feature can be uninstalled to convert the server to Server Core. In addition the MMC snap-ins for the server roles can be easily installed on client editions of Windows 8 using the Remote Server Administration Tools (RSAT), to facilitate remote management of Windows servers. In addition, we are introducing over 2,430 new PowerShell cmdlets to enable local and remote management.

Reducing the Disk Footprint with Features on Demand

By default, the files for all available Windows components are stored in a directory called the side-by-side component store, or WinSxS. The structure of this directory was carefully designed to provide a number of benefits. For example, installation media is never required to turn any Windows feature on or off. In addition, Windows updates can be applied in any order—and any update can be completely skipped if an administrator so desires. Enabling this functionality requires a version of every Windows component that can be installed to be stored in the winsxs folder, requiring disk space which is always at a premium. As growing numbers of virtual machines vie for space on relatively expensive high-performance disks, SANs, and SSDs, we saw the need to enable administrators to reduce the disk footprint of Windows Server “8.” At the same time, we wanted to maintain the high level of patch and deployment flexibility that administrators have grown accustomed to.

In Windows Server “8”, we have added the capability for administrators to completely remove unneeded roles and features from their installations. In previous versions of Windows, a feature can be either “Enabled” or “Disabled.” In Windows Server “8,” there is a new state called “Disabled with payload removed.” The new -Remove flag on the Uninstall-WindowsFeature PowerShell cmdlet will put a feature into this state which removes all of its files from the winsxs folder.

For example, you can completely remove Windows Explorer, Internet Explorer, and other dependent components (and all their associated files) from disk with the following PowerShell command:

Uninstall-WindowsFeature Server-Gui-Shell -Remove

When a feature is disabled with the payload removed, the files are actually deleted from the side-by-side store and completely removed from disk. Consequently, this feature cannot be reinstalled without providing an installation source specified on the Install-WindowsFeature cmdlet with -source. By default, Windows will download missing components from Windows Update if an Internet connection is available (this behavior can be disabled if desired). If Internet access is not available the Windows image file (install.wim) available on the DVD can be mounted to provide the installation source. It is also possible to specify a default list of installation sources via Group Policy.

The Features on Demand functionality is already available in the Windows Server “8” Beta. For example, version 3.5 of the .NET Framework is not included in the image you may have downloaded. Not requiring the .NET 3.5 installations enabled us to reduce the file size of the ISO image by approximately 300 MB. Similarly, the files for some “Server with a GUI” components—such as GUI management tools and some server roles—have been removed in a default Server Core installation.

For more information on both Features on Demand as well as how to use Windows PowerShell to convert from a Server Core installation to a Server with a GUI installation, please see the Windows Server Installation Options article on TechNet.

Reboot and Patch Reduction

As we shared in our previous blog posting, we have been doing a lot of work to minimize the frequency of reboots due to servicing. Accordingly, Windows Update has been designed so that features which are not installed are also not patched. By disabling frequently-serviced and infrequently-used optional features—such as the GUI components on most servers—using Server Core it is possible to achieve a 40-60% reduction in patches based on historical data.

The above chart shows how many months a Windows Server Core deployment would have been able to go without reboots since the release of the respective product. Applicable patches are security updates offered and recommended by Windows Update. These patches are offered simply because the files affected by the patch are installed on the system. Necessary patches are a subset of applicable patches which are called out in security bulletins (such as MS08-052) that explain which of the applicable patches actually need to be installed for a particular scenario.

For example, in the case of MS08-052, a patch is offered for GDI+ that fixes a vulnerability that could enable malicious remote code execution. However, the vulnerability is only exploitable in certain situations, namely when certain programs other than Windows Server itself is installed. An administrator could safely opt out of this particular update if he or she determined the patch could not actually be exploited, thus saving a reboot. By installing only necessary patches, the number of reboots can be further reduced.

It is possible to reduce reboots even more by opting to install only critical updates. Since the RTM of Windows Server 2008, a Server Core installation could have seen 26 months without reboots – roughly one reboot every two months –a 67% savings as compared to a Server with a GUI installation.

Summary

We built Windows Server “8” to provide the greatest flexibility and resource optimization for all datacenters and cloud environments. The Server Core deployment mode provides increased uptime, reduced maintenance, optimized disk space and memory runtime requirements, and offers faster, more efficient deployments than ever before. We would like to encourage IT professionals and customers to enable your datacenters or private cloud deployments to use Server Core and the Minimal Server Interface, in order to provide the best possible solution for next generation solutions and applications.

Windows Server “8” – Taking Server Application Storage to Windows File Shares

2012-03-15T11:10:00Z

Whenever I work with the Windows Server “8” storage, I get a huge smile as I think about what customers are going to be able to do and the excellent engineering that went into the features. Whether you’re using a block-based storage area network (SAN) or a file-based solution, we are heavily invested in both types of storage based on your input. In this blog, we are going to focus on our investments in file-based storage. Our teams worked together to innovate up and down the entire storage stack. From the way we flush data, to Storage Spaces, to Resilient File System (ReFS), to improvements in the Server Message Block (SMB) protocol and Cluster Shared Volume (CSV), to the support for SMI-S enabled storage devices, Windows Server “8” fundamentally changes how we think about storage architectures and solutions. It boils down to this – Windows Server “8” minimizes the Capex/Opex of storage– in some cases, dramatically. If you are involved with storage, you need to stop what you are doing and take the time to understand Windows Server “8” storage and rethink how you do things going forward.

In this post, we explore a new scenario enabled by Windows Server “8”: server application storage on file shares. It all started with a simple question, “Why can’t server applications take advantage of our file servers?” That’s when the program managers, developers and testers rolled up their sleeves, and broke the problem down into its parts and designed a comprehensive set of features to make it happen. I encourage you to download the beta and see for yourself.

Claus Joergensen and Jose Barreto, Principal Program Managers in our File Server team, authored this post.

--Cheers! Jeffrey

Background

Since the beginning, the Windows file server has primarily been used for storing end user data. Typical business scenarios are the user home share for non-shared data and team shares for collaboration. The Windows Server “8” file server introduces support for server applications, such as Hyper-V™ and Microsoft SQL Server, that can store live data on Windows file shares. For example, a user can configure a Hyper-V virtual machine with its configuration file, VHD files, and snapshot files stored on a Windows file share. The following screenshot shows a virtual machine configured with its VHD file stored on a Windows file share, with the Universal Naming Convention (UNC) path:

Enabling new scenarios

During planning for Windows Server “8”, our customers showed particular interest in the ability to store server application data on Windows file shares. For many small and medium business customers, this is an affordable alternative to SAN infrastructures because they can leverage Ethernet infrastructures and use industry-standard servers. Also, when file shares are created, customers can more easily manage the file shares because they do not need to provision LUNs or configure zones. In addition to being a more affordable alternative and simpler to manage, the ability to store server application data allows larger customers and hosters to gain more flexibility when moving workloads around in the datacenter without having to reconfigure storage. If data is stored on a UNC path, with the right administrative credentials, it can be accessed from anywhere in the datacenter.

With support for this new scenario, all customers have an additional storage option and can choose between Fibre Channel, iSCSI SANs, shared SAS storage arrays, or file shares, depending on their preferences, budgets, and required features.

Enabling server application storage on file shares

To enable server applications to store their live data on file shares, there are two requirements. First, the server role or application needs to support it. This includes updating the application to support UNC paths (\\server\share\file.vhd) in its setup and management tools, as well as fully testing the applications in the use cases in this scenario. In Microsoft SQL Server 2008 R2, there is added support for storing SQL user databases on SMB file shares. Microsoft SQL Server 2012 adds support for the SQL system database, as well as configuring SQL Server as a cluster. As demonstrated at the //BUILD conference, Windows Server “8” has also added support for storing virtual machines files on SMB file shares.

Second, the file server itself needs to support allowing server applications to store their data on file shares. During Windows Server “8” customer planning engagements, we identified the following top-level requirements for the file server to support storing server applications:

- Continuous availability. Server applications expect storage to always be available and in general, do not handle input/output (I/O) errors or unexpected closures of file handles well. These types of events may cause virtual machines to crash because the virtual machine can no longer write to its disk or cause databases to go offline. Customers commonly deploy hardware redundancy, such as multiple network adapters, network switches, and Windows cluster configurations to mitigate hardware outages. While such configurations allow the file server to quickly recover from a failure, the recovery is not transparent to the application and virtual machines must be restarted and databases brought online. The Windows Server “8” file server solution must be able to quickly and transparently recover from network or node failures, with no downtime or administrator intervention required.

- Performance. Some server roles, such as Hyper-V and SQL Server, are very sensitive to storage performance, including bandwidth, latency and I/O per second (IOPS). It is also important to ensure that CPU consumption when accessing storage is kept to a minimum to provide as much CPU time to the application as possible. Finally, server applications tend to have an access pattern that is very different than that of user applications. Where user applications mostly read or write a file in full, server applications tend to append or update existing data. The Windows Server “8” file server solution must be able to deliver storage bandwidth to server applications almost equivalently to that of multiple 10Gbps Ethernet network or Infiniband adapters with latency, IOPS, and CPU consumption rivaling that of Fibre Channel.

- Scalability. The configurations for a Windows file server cluster are often deployed in active-passive configurations, which leaves at least one node unused. A workaround is to configure multiple file server instances in a cluster. This allows you to use all of the hardware in the cluster. However, this requires additional administration and the bandwidth available for a share is still limited to the bandwidth available on the node where it is currently online. The Windows Server “8” file server must be able to support active-active configurations where a share can be accessed through any node, increasing the maximum bandwidth to the aggregate of the cluster nodes and simplifying administration.

- Data protection. Another key ability is creation of application-consistent shadow copies of the data for backup purposes. In Windows, this is usually accomplished using the Volume Shadow Copy Service (VSS) infrastructure. VSS, in its current form, only supports local storage. The Windows “8” file server solution must be able to support application consistent shadow copies through full integration with VSS and with minimal impact on existing VSS requestors, writers, and providers.

As you can see, this is quite a demanding list of requirements. However, we agreed that we needed to address all of them to provide a reliable, available, and serviceable file server with great performance for server application storage.

Features overview

Supporting server application storage on file shares in Windows Server “8” was a major decision for the product team. Several features were introduced specifically to make sure file storage could meet or exceed the requirements commonly applied to block storage, without losing file storage’s inherit benefits in ease of management and cost effectiveness. This also required the introduction of a new version of SMB, which is Window’s main remote file protocol. These new capabilities include:

SMB Transparent Failover: Enables administrators to perform hardware or software maintenance of nodes in a clustered file server without interrupting server applications storing data on these file shares. Also, if a hardware or software failure occurs on a cluster node, this feature enables SMB clients to transparently reconnect to another cluster node without interrupting server applications that are storing data on these file shares. This is achieved regardless of the type of operation that is under way when the failure occurs. For block-based storage, this is the equivalent of having a multi-controller storage array.

SMB Multichannel: Enables you to simultaneously use multiple connections and network interfaces with two main benefits: increased throughput and fault tolerance. For instance, if you have four 10GbE interfaces on both the SMB client and server, you can simultaneously use them to effectively achieve 40Gbps throughput from the four 10Gbps network adapters. In the event that one of the network adapters or cables fails, your SMB client will continue to use the network uninterrupted, at a lower throughput. Best of all, this is achieved without additional configuration steps. You only need to configure the multiple network interfaces as you normally would.

SMB Direct: One of the main advantages of Fibre Channel block storage is the ability to have low latency and fast, offloaded data transfers. To match that in the file server world, SMB introduces support for network adapters that have RDMA capability and can function at full speed with very low latency, while using very little CPU. When using one of three RDMA technologies (Infiniband, iWARP or RoCE), the SMB client has a low CPU overhead, which is comparable to Fibre Channel, and saves CPU cycles for the main workload on the box, such as Hyper-V or SQL Server. Best of all, these network interfaces are detected and function without requiring additional SMB configuration steps. If RDMA interfaces are available, they will be automatically used.

SMB Scale-Out: Taking advantage of Cluster Shared Volume (CSV) version 2, administrators can create file shares that provide simultaneous access to data files, with direct I/O, through all nodes in a file server cluster. This means that the maximum file serving capacity for a given share is no longer limited by the capacity of a single cluster node, but rather the aggregate bandwidth across the cluster. Also, this active-active configuration lets you balance the load across cluster nodes by moving file server clients without any service interruption. Finally, SMB Scale-Out simplifies the management of clustered file servers and file shares.

VSS for SMB File Shares: The ability to create application-consistent snapshots of the server application data is critical to backing up the data. In Windows, this is accomplished using the Volume Shadow Copy Service (VSS) infrastructure. VSS for SMB file shares extends the VSS infrastructure to perform application-consistent shadow copies of data stored on remote SMB file shares for backup and restore purposes. In addition, VSS for SMB file shares enable backup applications to read the backup data directly from a shadow copy file share rather than involving the server application computer during the data transfer. Because this feature leverages the existing VSS infrastructure, it is easy to integrate with existing VSS-aware backup software and VSS-aware applications, such as Hyper-V.

SMB-specific Windows PowerShell cmdlets: Managing file shares is now accomplished using either the new Windows Server Manager GUI supporting file server clusters, which includes several profiles for creating SMB shares, or using the all new SMB Windows PowerShell cmdlets, which use the familiar Windows PowerShell infrastructure for command-line and scripting. This complete new set of Windows PowerShell version 3 cmdlets was created to manage file shares, file share permissions, client mappings, server configuration, and client configuration. There is also an extensive set of cmdlets to monitor sessions, open files, connections, network interfaces, and multichannel connections. These cmdlets are built upon a standards-based management protocol using WMIv2 classes that allow developers, on Windows and Linux, to create automated solutions for file server configuration and monitoring.

SMB Performance Counters: In the application server world, storage performance is paramount, as is the ability to measure it. With that in mind, Windows Server “8” includes server and client performance counters that allow administrators to easily look into the key metrics for file storage, including IOPs, latency, queue depth, and throughput. These counters match the familiar block storage performance counters, making it simple to leverage your existing skills and guidance for storage performance for Windows Server.

Performance: Performance was also a key area of focus in SMB. In addition to making the large maximum transmission unit (large MTU) enabled by default, there was a significant amount of work to optimize performance for different kinds of workloads, covering both small and large I/O, and both sequential and random access. These optimizations were developed while investigating typical end-to-end workloads, such as online transaction processing, data warehousing, virtual web servers in a private cloud, virtual desktop infrastructure, and consolidated home folders. These investigations led to specific improvements in many areas of the operating system.

Let us take a closer look at SMB Transparent Failover. SMB Transparent Failover requires:

A failover cluster running Windows Server “8” with at least two cluster nodes and configured with the file server role. The cluster must pass the cluster validation tests in “Validate a Configuration Wizard”.
File shares created with the continuous availability property, which is the default setting for clustered file shares.
Computers accessing the clustered file shares must be running Windows “8” Consumer Preview or Windows Server “8”.

When the SMB client initially connects to the file share, the client determines whether the file share has the continuous availability property set. If it does, this means the file share is a clustered file share and supports SMB transparent failover. When the SMB client subsequently opens a file on the file share on behalf of the application, it requests a persistent file handle. When the SMB server receives a request to open a file with a persistent handle, the SMB server interacts with the Resume Key filter to persist sufficient information about the file handle, along with a unique key (resume key) supplied by the SMB client, to stable storage. The SMB client uses the resume key to reference the handle during a resume operation after a failover. To protect against data loss from writing data into an unstable cache, persistent file handles are always opened with write through.

If a failure occurs on the file server cluster node to which the SMB client is connected, the SMB client attempts to reconnect to another file server cluster node. Once the SMB client successfully reconnects to another node in the cluster, the SMB client starts the resume operation using the resume key. When the SMB server receives the resume key, it interacts with the Resume Key filter to recover the handle state to the same state it was prior to the failure with end-to-end support (SMB client, SMB server and Resume Key filter) for operations that can be replayed, as well as operations that cannot be replayed. The application running on the SMB client computer does not experience any failures or errors during this operation. From an application perspective, it appears the I/O operations are stalled for a small amount of time.

It is very important to keep the number of I/O stalls during a failover to a minimum. Since SMB is sitting on top of TCP/IP, the SMB client would normally rely on TCP timeout to determine if the file server cluster node has failed. However, relying on TCP timeouts can lead to fairly long I/O stalls, since each timeout is typically ~20 seconds. SMB Witness was created to enable faster recovery from unplanned failures, allowing the SMB client to not have to wait for a TCP timeout. SMB Witness is a new service that is installed automatically with the failover clustering feature. When the SMB client initially connects to a file server cluster node, the SMB client notifies the SMB Witness client, which is running on the same computer. The SMB Witness client obtains a list of cluster members from the SMB Witness service running on the file server cluster node. The SMB Witness client picks a different cluster member and issues a registration request to the SMB Witness service on that cluster member.

If an unplanned failure occurs on the file server cluster node, the SMB Witness service on the other cluster member receives a notification from the cluster service. The SMB Witness service also notifies the SMB Witness client, which in turns notifies the SMB client that the file server cluster node has failed. Upon receiving the SMB Witness notification, the SMB client immediately starts reconnecting to a different file server cluster node, which significantly speeds up recovery from unplanned failures.

Deployment modes

When planning Windows Server “8”, from an end-to-end perspective, the two main areas of focus for file storage for server applications are Hyper-V over SMB and SQL Server over SMB.

For example, when using Hyper-V, SMB file storage is now fully supported for both standalone and clustered configurations of Hyper-V. In fact, it is now possible to configure a Hyper-V cluster using only file shares as the shared storage.

For the file server configuration, there are three main modes for deployment:

Even though, the single-node or standalone file server is not highly available, it provides the most inexpensive file server solution. Hyper-V includes added flexibility of shared storage and allows you to cluster the Hyper-V nodes using this solution (although the overall solution should not be considered highly available). When compared to block storage, this is the equivalent of a single-controller block storage array.

The dual-node file server is expected to be the most common file server configuration, providing continuous availability (via SMB transparent failover) at a low cost. Using shared Serial Attached SCSI (SAS) storage (just-a-bunch-of-disks [JBOD] s or a SAS-based Storage Array), this solution can scale to a few hundred disks. Paired with a few Hyper-V servers and network switches, this solution could be used to create a rack-sized building block for a private cloud solution. This would be the equivalent of a dual-controller block storage array.

The third option, with a larger number of file servers using Fibre Channel as shared storage, allows for larger configurations. This file server cluster can leverage features like SMB Scale-Out and SMB Direct to create a shared storage infrastructure to serve dozens, if not hundreds, of Hyper-V nodes. There are significant savings in limiting the Fibre Channel connectivity to the file server nodes when using a 10GbE or InfiniBand to connect the Hyper-V computer nodes to the file servers.

Conclusion

We are very excited by the new opportunities for file server storage with server applications. In fact, we have very positive feedback early adopters, both internally and externally, who have agreed to deploy these scenarios and features. They share our enthusiasm for the ease of use, manageability, scalability, and performance of the solution.

If you would like to learn more, we have a series of presentations that were delivered during the //Build/ conference in September, 2011. These presentations are available here, and they provide hours of additional details recorded in video. We encourage you to review them. Here is a list of the main sessions on Windows Server “8” related to this blog post:

I hope you enjoy watching them as much as we enjoyed putting them together…

Claus Joergensen and Jose Barreto, Principal Program Managers in the Windows File Server team.

Rocking the Windows Server “8” Administrative Experience

2012-03-07T09:45:00Z

Howdy! My name is Jeffrey Snover. I am the Distinguished Engineer for Windows Server where I help drive the architectural direction and technical strategy of the product. It’s possible you know me as the inventor of Windows PowerShell. As we continue to use this blog to introduce and explain the new capabilities in Windows Server “8,” I will be introducing the team members and their posts. I’ll also be writing some of the posts myself. Think of me as your host and guide as we help you explore Windows Server “8” with this blog. We’ll start with something near and dear to my heart: Admins.

Windows Server has always distinguished and prided itself on its Administrative Experience. Our mission for Windows Server “8” was to deliver the best cloud-optimized OS. This required us to reimagine the experience, focusing on scenario-based multi-machine management from a client machine implemented on top of PowerShell and WMI. When you see and use it, I think you’ll agree that it is clean, powerful, intuitive and just plain fun. The architecture ensures that everything you can do from the GUI, can also be automated from the command line. Automation drives up your server-to-admin ratio, increases the quality and repeatability of your IT operations and lets you schedule operations on the weekend while you are enjoying your time off. Ultimately the Administrative Experience is all about making people successful. This blog highlights a few of the many changes we made in Windows Server “8.” Read about them here and then download the beta of Windows Server “8” and the Remote Administration Tools and try them for yourself. I think you are going to have some fun.

Erin Chapple, a Partner Group Program Manager on our Windows Server Manageability team, authored this post.

--Cheers! Jeffrey

__________________________________________________________________________________________________________________________________________________________________

One of Windows Server’s key differentiators has always been our focus on providing an Administrative Experience that leads the industry in enabling administrators to manage their servers. Windows Server “8” takes this to a whole new level with substantial improvements in simplicity, richness and the power it provides to administrators.

Redesigned Server Manager and Integrated Experiences

Two releases ago, we introduced Server Manager. Server Manager provided a cohesive role-centric view of a single server, exposing the common management tasks administrators performed on a daily basis. When we looked at challenges facing our customers, it was clear that Server Manager was a good start, but that with Windows Server “8” we needed to reimagine the experience. And so we took on the task of redesigning Server Manager to deliver the experience needed for a cloud-optimized OS.

Multi-Server

As we move to the cloud, one of the key shifts needed in Server Manager was from the single role-centric view to a multi-sever view of the environment. Server Manager delivers a multi-server experience enabling administrators to add the servers they are responsible for, view information (such as events, services, performance) across their servers and take action. In addition, the Administrative Experience is consistent across servers whether they are physical or virtual. Server Manager accomplishes this by leveraging the multi-machine management capabilities of WMI, Windows PowerShell and Windows PowerShell’s new workflow capabilities. Virtually every operation done using Server Manager can also be done via Windows PowerShell. This allows admins to automate operations thereby saving time, increasing quality and consistency and improving server-to-admin ratios.

Fresh Yet Familiar Experience

With the design movement across Microsoft to Metro style, it was clear we had an opportunity to modernize the Administrative Experience. The battleship grey that server can be identified with needed to be refreshed and brought forward. And yet, we know that any new experience needed to be connected with how administrators get things done today, so as not to disrupt their work patterns. As a result, we looked at the Metro Style Design Principals and centered on three areas of focus for the new Server Manager:

· Glance-able – The administrator can understand through a quick glance the state of their environment and where they need to focus their attention.

The Server Manager dashboard provides a glance-able view of the server environment, drawing attention to the key issues needing attention.

· Actionable – The administrator can take action directly based upon the information presented to them. No need to open another tool, just a simple click resolves the issue.

From the Server Manager dashboard, the user can view the Services that are not stopped and start the service across multiple machines.

· Relevant – The administrator can to tailor the experience to their needs. The information presented should be customizable based upon their environment and responsibility to provide just what they need.

From the Manage menu in Server Manager the administrator can add a custom group to display on the dashboard.

The custom server group will appear as a tile on the dashboard and the administrator can then understand the state of this group.

Integrated Scenario-based Experiences

Server Manager provides a set of consistent tasks across servers, allowing the administrator to drill-into server-specific views to understand the state of their environment and take action. Server-centric views are only one pivot necessary for effective management. Role-centric views are equally important, and with the newly redesigned Server Manager there are several server roles that extend Server Manger to provide scenario-based experiences for managing their role.

File Services, Remote Desktop Services and IP Address Management have all delivered new Administrative Experiences that follow the Server Manger design principles outlined above. The result is a combined experience that is contextual to what the administrator is managing and both guides the user through specific tasks while also providing connected information that is helpful in troubleshooting problems.

The combination of the above scenarios and the shift to multi-machine management means a significant increase in the data exposed within Server Manager. Consuming this information requires enhanced capabilities. Throughout the new Server Manger experience you will see rich filtering and pivoting capabilities for the administrator to find, organize and act on the data provided.

Consistent through Server Manager is the ability to filter lists to easily find information, organize and take action.

Support for Previous Version of Windows Server

Our administrators live in a world where they manage multiple versions of Windows Server. To support providing one view of their server environment we have created a set of new WMI providers that allow Server Manager to collect information from Windows Server 2008 R2 and Windows Server 2008 machines. The providers are available in the Windows Management Framework 3.0 and when installed on a Windows Server 2008 R2 or Windows Server 2008 machine, Server Manager then collects details such as the events and services from these machines and aggregates into the dashboard.

Before we go any further, let’s be clear – we believe in delivering rich GUI experiences that are tailored to administrators and help them accomplish their work easily and efficiently. GUIs are here to stay. The reimagined Server Manger and integrated tools for Windows Server “8” are best of breed for the tasks they expose.

AND the Administrative Experience isn’t just about GUIs.

AND servers should be used for server-related tasks, not as an administrative desktop.

Let’s dig into each of these in some detail.

Command-line Interface (CLI) is part of the Administrative Experience

A number of people have misinterpreted our investment in Windows PowerShell to indicate a transition to a CLI world. They present this is a GUI VERSES CLI issue. We’ve never thought in those terms. We have always viewed Windows PowerShell as additive, so we’ve always viewed this as a GUI AND CLI issue.

There will always be administrators who prefer to use the GUI. That said, there are tasks made more effective through the use of automation and we provide as rich a CLI experience as we do a GUI experience for these customers. More importantly, there are advantages of using an automation solution that are becoming more and more important with the move to cloud and the corresponding scale that we expect from administrators. Namely, automation removes the human factor, increasing reliability, auditability and predictability in the environment. Automation is a whole post unto itself (watch for it!) so we’ll focus on the CLI experience improvements for now, namely the investments we’ve made in making Windows PowerShell Integrated Scripting Environment (ISE) a great onboarding tool for Windows PowerShell.

Here are three of my favorite new features in Windows PowerShell ISE that are targeted at helping you discover, learn and simplify the automation of your servers.

Show Command

With over 2,300 cmdlets in Windows Server “8”, the first question on your mind might be – how do I discover the cmdlet I need to get my job done? The new Show Command windows in Widows PowerShell ISE lets you easily search for cmdlets, discover the parameters and then either run the command or insert it into a script. Show Command takes advantage of the unique architecture of Windows PowerShell where each cmdlet declares its parameters and their metadata and Windows PowerShell provides a single common parser for every cmdlet. This architecture allows Show Command to search cmdlets and use the parameter metadata to generate a GUI interface for the cmdlet. You can have some fun exploring this part of the architecture by typing the following command into Windows PowerShell: (Get-Command Get-Process).ParameterSets

The Show Command window allows the administrator to search the cmdlets available, learn the syntax and either run, insert or copy the command.

Intellisense

Now that I’m familiar with the new cmdlets, the next question is – how do I remember the cmdlets when I’m working? The promise of Windows PowerShell has always been that the consistency of implementation allows you to Think, Type and Get what you need. Intellisense lets you take this to the next level, by exposing the syntax of the cmdlet as you type. As you build up the command, it only shows you those parameters that are consistent with the parameters that you have already selected. Intellisense can almost seem like magic. It can do this because Windows PowerShell V3 uses the .NET Dynamic Language Runtime (DRL) and exposes a public Abstract Syntax Tree (AST) which allows Intellisense to reason about the command line and the context that it is executing in.

The new Intellisense functionality in the Windows Powershell ISE helps administrators discover cmdlets and their syntax.

Intellisense doesn’t just work on the syntax of cmdlets it understands the input needed, here providing insight into the file system so the administrator doesn’t need to remember the path!

Snippets

Now that I’m ready to put my newfound knowledge of the cmdlets into action in a script – how do I remember the syntax for common scripting tasks? Turning on snippets gives me access to common scripting patterns with the click of a mouse! Snippets are also fully extensible which means I can add my own common scripting patterns to avoid have to remember (and type) the pattern each time I write a new script.

After turning on Snippets (Ctrl + J) the administrator can choose from a set of built-in scripting patterns.

In this case the administrator chose the if-else scripting pattern and it is populated for the administrator in the script window.

Through these improvements we’ve significantly increased the approachability of scripting thereby exposing the power of automation to a much broader audience.

Server Core is the Preferred Deployment Option

While we love GUIs, we believe the primary place they should exist is on the administrator’s desktop – not on the Server! Server resources are much more expensive than client resources and running GUIs on servers requires additional software components. Every component increases the security and serviceability exposure of that server so you should only install those components that are necessary to that server workload. Fewer things running on the server means fewer patches and more resources available to the server workload. In this release we’ve made several investments to help administrators succeed in choosing Server Core as the primary deployment option for Windows Server. The traditional “Server with a GUI” is still provided as a backwards compatibility option.

The number of server roles that run on Server Core has increased to 13 with support for SQL 2012, eliminating the most common reason administrators cited for not being able to run in the Server Core configuration. Firewall-friendly remote management (WinRM) and Windows PowerShell are now enabled and installed by default on all servers, removing any configuration needed before being able to manage the server remotely. Windows PowerShell’s 2300+ cmdlets provide the command line coverage necessary for most admin scenarios. For the first time ever, we released a Beta version of the Remote Server Administrative Tools at the same time as the Server Beta providing a rich GUI experience to manage all Servers, including Server Core, from a Windows Client.

Perhaps most significantly however, we’ve added the ability to move between Server Core and “Server with a GUI” without the need to reinstall the server! This means administrators can safely start with their server deployed in the Server Core configuration and if they find they need the GUI they can add it, and also remove it as needed using the SCONFIG CLI tool, Windows PowerShell or the Add/Remove Roles and Features Wizard. Stay tuned for a future blog dedicated to Server Core that will provide you with all the benefits and details of this deployment option.

The default experience for Server Core.

And if that wasn’t enough, introduced in a blog post from earlier this year, we’ve added a Minimal User Experience option which allows GUI tools to run on Server Core but does not install the desktop shell or Internet Explorer. Server Manager and cmd.exe launch by default when you log in and you can use these to launch the other GUI tools. This in-between option provides many of the benefits of Server Core while still having the safety-factor of being able to run GUIs should the administrator need to log into the Server directly.

From the Manage menu in Server Manager the administrator can select Remove Roles and Features to move between Server with a GUI and the Minimal User Experience or Server Core.

The Minimal User Interface provides many of the benefits of Server Core but lets administrators run GUI tools like Server Manager and MMC-based tools like Computer Manager.

Oh, heads up – Server Core is the default (and recommended) selection during installation!

Metro Style and the Local Server Experience

We can’t finish a blog post on the Administrative Experience without discussing how Windows Server shares the new design language and capabilities of the Metro style interface. Early in the design of Windows Server “8” we talked to lots of administrators to understand what direction they wanted us to pursue. The feedback we received was that most administrators manage both server and client machines. As a result, consistency was extremely important. Consistency doesn’t just apply to our administrators. In order to support end-users having the same desktop experience whether local, or using Remote Desktop Services, it was important to share the shell experience and adopt the new Start Screen. We have customized the default experience on the server to optimize for administrative tasks. To this end, we default to the desktop on logon. We bring up Server Manager by default. Server Manager exposes the full set of administrative tools via its Tools menu. We pin common applications like Server Manager and Windows PowerShell to the Task Bar. When you go to the Start Screen you’ll similarly see common applications exposed.

By default, when logging onto the Server, Server Manager starts. Additional Administrative Tools are available from the Tools menu in Server Manager and common tasks like Windows PowerShell and Explorer are pinned to the Task Bar.

The Start Screen in Server has common administrative tools pinned by default.

Given the recommended deployment option of Server Core, and the introduction of the Minimal GUI Interface, we hope that most administrators will rarely find themselves using the Start Screen on the server, but when they do it will be easy for them to find what they need to do in a manner consistent with the client experience. As mentioned earlier, stay tuned for a future blog detailing Server Core and the Minimal GUI Interface.

In conclusion, we have invested significantly in the Administrative Experience for Windows Server “8” to ensure administrators have choices for how they complete their work and that each choice is simple and efficient. Windows Server “8” will provide the fastest, most scalable and flexible solutions for customers large and small.

Give it a try and enjoy the next generation server management experience!

Where to Find Previous Windows Server “8” Posts

2012-03-07T02:00:00Z

The Windows Server Product team is streamlining the blogging process. All future Windows Server Blogs will be coming from here. You will hear a lot more over the next couple of months from Bill Laing, VP of the Server and Cloud Division, Jeffrey Snover, Distinguished Engineer for Windows Server and other leaders from Windows Server. We will also be trying to give you links to a lot of even deeper content on Windows Server “8” coming from our Product Team Blogs. As most of you know, we have a number of very knowledgeable engineers who write a lot of great information on our supporting technologies, like Hyper-V, RDS, File and Storage, Group Policy, etc… So, please stay tuned because we plan to bring you a lot of great information on Windows Server ‘8”.

As you know, the beta of Windows Server “8” is now available. So, please go download, evaluate, and give us feedback. Please also take some time to let us know what you think, ask us questions and suggest ideas for new content that hasn’t been written yet. We love comments. J

Here is a complete list of all the past Windows Server "8" posts since the Build Conference in September 2011:

We look forward to sharing as much as we can with you over the next couple of months.

Sincerely,

The Windows Server Core Blogger Team

Windows Server “8” beta available now!

2012-03-01T16:00:00Z

The beta of Windows Server “8” is now available for IT professionals and software developers around the world to download, to evaluate, and to give us feedback on.

In September we introduced Windows Server “8” with a preview to help developers and hardware partners prepare new and existing applications, systems and devices. The response from that community, along with hundreds of customers in our early adopters program, has been incredibly positive. A common theme of feedback has been how broad and deep the new capabilities are.

Now is the time for you, IT professionals in organizations of all sizes, to get your hands on this new release, discover the new capabilities and contribute to the development of what we call the cloud-optimized OS.

I’ll highlight in this post just a few examples of new capabilities that you’ll want to explore.

With the new Hyper-V we are taking virtualization above and beyond to provide a multi-tenant platform for cloud computing. For example, with Hyper-V Network Virtualization you can create virtual networks so different business units, or even multiple customers, can seamlessly share network infrastructure. You will be able to move virtual machines and servers around without losing their network assignments.

In Windows Server “8” we are delivering high availability and disaster recovery through software technology on much more cost effective hardware. For example, with File Server Transparent Failover you can now more easily perform hardware or software maintenance of nodes in a File Server cluster by moving file shares between nodes with little interruption to server applications that are storing data on those file shares.

We’re also delivering a tremendous amount of new capabilities for multi-machine management and automation. You will want to explore the dramatic new improvements to Server Manager, as well as the new Windows PowerShell. With 2,300 commandlets provided out of the box, Windows PowerShell allows you to automate everything you can do manually with the user interface. And, with technologies like Intellisense, we’ve made it very easy for you to master all of that power.

Additionally, Windows Server “8” provides a powerful server application platform that enables you to develop and host the most demanding of application workloads. For example, with .NET Framework 4.5 you can take advantage of new asynch language and library support to build server and web applications that scale far beyond what other platforms provide. Our new IIS 8 web server provides better security isolation and resource sand-boxing between applications, native support for web sockets, and the ability to host significantly more sites on a server.

This is just a brief taste of the hundreds of features and capabilities you will find in the beta. (My team has written a number of other posts you can read here.) If you have been using and providing feedback on the developer preview of Windows Server “8,” thank you! I can’t wait for more people to start trying out Windows Server “8” and letting us know what they think.

Bill Laing
Corporate Vice President, Server and Cloud

Windows Server 8: Standards-Based Storage Management

2011-10-14T18:45:00Z

Take a look at Jeffrey Snover's blog post, Distinguished Engineer and Lead Architect for Window Server, where he discusses standards-based storage management in the next release of Windows Server codenamed "Windows Server 8".

Read Jeffrey's post on the Microsoft Server and Cloud Platform blog.

Storage and Continuous Availability Enhancements in Windows Server 8

2011-09-20T17:10:00Z

Check out Thomas Pfenning's blog post, General Manager Server and Tools, where he discusses the storage and availability enhancements in the next release of Windows Server codenamed "Windows Server 8".

Read Thomas' post on the Microsoft Server and Cloud Platform blog.

Windows Server 8: An Introduction

2011-09-11T17:27:00Z

Take a look at Bill Laing's blog, Corporate Vice President for Microsoft's Server and Cloud business, to get an overview of the next release of Windows Server codenamed Windows Server 8.

Read Bill's post on the Microsoft Server and Cloud Platform blog.

Customers Reap Benefits from Comprehensive Cloud Approach

2011-08-29T15:20:00Z

Take a look at Brad Anderson’s, Corporate Vice President at Microsoft, perspective on Microsoft’s cloud computing strategy, our private cloud solutions and the economics of those solutions versus VMware.

Click here to read more.

Check out the New Microsoft Server and Cloud Platform Website!

2011-08-25T20:14:19Z

We’re pleased to announce the launch of the new Microsoft Server and Cloud Platformwebsite
(http://www.microsoft.com/server-cloud).

The new site is a comprehensive source for information relating to our private cloud
and virtualization solutions, as well as other solutions based on Windows
Server, System Center, Forefront and related products. It is designed to
help connect you to relevant information across Microsoft web destinations,
including TechNet, MSDN, Pinpoint and more - making it a great place to
understand what we offer, why you should consider it and how to get started.

Enjoy the new site and stay tuned to the blog for the latest Server & Cloud
Platform updates!