In the first of a two part series, I provide some background information about PowerShell and DevOps. In the second blog, I’ll provide you a bunch of specifics. PowerShell 3.0, like Windows Server 2012, as a ton of new features and enhancements so I’ll only scratch the surface.
--Jeffrey
The first time I heard DevOps was a podcast describing the 2009 Velocity conference. While most of the industry was struggling to deploy releases a few times a year, John Allspaw and Paul Hammond rocked the house with the talk “10 Deploys Per Day: Dev And Ops Cooperation at Flickr”. They made the case for delivering business results through changes in culture and tools, and gave birth to a new term: DevOps. The problem is that developers think they are responsible for delivering features and operators are responsible for keeping the site running. The gap between developers and operators leads to finger-pointing when things go wrong. Successful business requires an IT culture of joint accountability and mutual respect: developers thinking about the needs and concerns of operators and operators thinking about the needs and concerns of developers. Their talk described how businesses required rapid change but that change is the root cause of most site-down events. Shunning the traditional “avoid change” approach, they advocated minimizing risk by making change safe through automation. This is the job of DevOps – safe change. This was the Taguchi quality approach applied to IT operations. Taguchi observed that the root cause of poor quality was variation. The solution was to first figure out how to do something repeatably. Once you could do that, then you can make small modifications in the process to see whether they make things better or worse. Back out the changes that make things worse. Keep doing the things that make things better. The key is repeatability. Repeatability allows experimentation which drives improvement. We get repeatability in IT operations through automation.We started PowerShell by publishing the Monad Manifesto which articulated the problems we saw, our approach to solving them and the components we would deliver. We envisioned a distributed automation engine with a scripting language which would be used by beginner operators and sophisticated developers. PowerShell’s design was driven by the same thinking and values that drove the birth of DevOps:
Focus on the businessPowerShell has always focused on people using computers in a business context. PowerShell needed to be consistent, safe, and productive. Much has been made of the similarities between PowerShell and UNIX but in this regard, our ties are much closer to VMS/DCL and AS400/CL.Consistent: Operators and developers don’t have a lot of time to learn new things. A consistent experience lets them to invest once in a set of skills and then use those skills over and over again. PowerShell uses a single common parser for all commands and performs common parameter validation delivering absolute consistency in command line syntax. PowerShell cmdlets are designed in a way that ubiquitous parameters can provide consistent functions to all commands (e.g. –ErrorAction, –ErrorVariable, –OutputVariable, etc)Safe: An Operator once told me that occasionally he was about to do something and realized that if he got it wrong, he would be fired. In PowerShell, if you ever execute a cmdlet which has a side-effect on the system, you can always type –WhatIf to test what would happen if you go through with the operation. We also support –Confirm, -Verbose and –Debug. Despite these safeguards, things can go wrong and when they do, PowerShell spends a lot of effort to speed up the process of diagnosing and resolving the error.Productive: Every aspect of PowerShell’s design maximizes the power of users (ergo the name). PowerShell makes it easy to perform bulk operations across a large number of machines. PowerShell also makes it easy to have productive engagements between your operators and developers because it allows them to speak a common language and to help each other with their scripts.Make change safe through automationThere has been a lot of discussion about whether PowerShell is a .Net language, a scripting language, or an interactive shell. PowerShell is a distributed automation engine with a scripting language and interactive shell(s). Interactive shells and a scripting language are critical components but the focus has always been on automation through scripting. Automation is the process of reducing and/or eliminating operations performed by a human. A script documents what is going to happen. People can review a script and you can modify it based upon their feedback. You can test the script, observe the outcome, modify the script and if modification is good, keep it and it if is bad back it out. In other words, scripting provides the repeatability required to apply the Taguichi method to IT operations. Once you have an automated process, you can safely apply it over and over again. These processes can now be performed reliabily by lower skilled admins. These steps aren’t possible when you use traditional GUI admin tools.Bridge the gap between developers and operatorsOur goal has always been to deliver a single tool which could span the needs of operators doing ad hoc operations, simple scripting, formal scripting, advanced scripting and developers doing systems-level programming. PowerShell spends a ton of effort trying to project the world in terms of high level task-oriented abstractions with uniform syntax and semantics. We call these cmdlets. And this is what operators want to efficiently and effectively manage systems. In order to copy a file using APIs, you would do this:
Have you ever wondered why PowerShell uses curly braces {} (and other C constructs) instead of BEGIN/END as other scripting languages do? We did that because we wanted to make it easier to adopt by developers of other C-based programming languages: C++, Objective C, Java, JavaScript, Perl, PHP, etc. We did some testing and determined that operators were able to readily adapt to this syntax. We also wanted to provide a smooth glide path between PowerShell and C# . This provides career mobility for operators who might want to transition to being a developer.Most importantly, we wanted to develop a tool which could be used by BOTH operators and developers to bridge the gap between the groups and allow them to create common scripts, learn from each other and work together.Windows Server 2012 and PowerShell 3.0 are excellent DevOps toolsDevOps is a new term and there is some disagreement about what it entails but at the heart it is all about making change safe through automation and bridging the gap between operators and developers. There is a lot to do in this area but Windows Server 2012 and PowerShell 3.0 make excellent progress towards accomplishing those goals. PowerShell won’t be the only tool in your DevOps toolbox but it should be in every DevOps toolbox. Download the beta today and find out for yourself.Cheers!
Jeffrey SnoverDistinguished Engineer and Lead Architect for Windows Server
We’ve put a lot of work into Windows Server 2012, and are happy to see the positive feedback from press and those who’ve downloaded the beta. We look forward to presenting Windows Server 2012 at our June TechEd events in Orlando and Amsterdam. In this blog, Christa Anderson, a Program Manager in our Partner and Customer Ecosystem team, talks about an opportunity for those that can’t make it to those events.
-Cheers! Jeffrey
Not everyone who wants to learn in-depth about Windows Server 2012 can make it to TechEd. Therefore, in cooperation with our co-sponsors Dell and HP, we’re sponsoring the Windows Server 2012 Community Roadshow, a complimentary series of events held around the world to present on some of the key infrastructure improvements we’ve made in Windows Server 2012, including:
Even better, this road show will be presented by Microsoft MVPs: independent technology experts and experienced presenters. MVPs speak regularly at third-party and Microsoft industry conferences such as TechEd, so we’re thrilled to have so many MVPs presenting in this road show.You can sign up at no cost for technical sessions in your area from the Windows Server 2012 Community Roadshow web site. Some events are already full, but you can get on the wait list by clicking the SOLD OUT link and following the instructions.This is a great opportunity to learn about core infrastructure improvements in Windows Server 2012 from independent technical experts. Go register!
Christa Anderson
We constantly strive to reduce the steps required for you to get your job done. One of the reasons Windows Server 2012 is a such great release is that we spent so much time listening to our customers and understanding their scenarios and concerns. When development teams start from a technology/feature mindset, it can be hard to work across groups because helping another team usually means that you have to give up something you wanted to do. We were able to achieve a very high level of technology integration and cross-group cooperation because we all shared a common understanding of our customers and their scenarios. Teams were eager to help each other succeed in delivering those scenarios. When you have lots of teams working together towards a common goal, you can really change the game and tackle some really hard problems. Today’s blog is a good illustration of that.Anyone that has been involved in securing data or accessing data security knows that the traditional security models and mechanisms are not always flexible enough to address today’s concerns and scenarios. Whether it's compliance requirements, increased business impact of disclosed data, or management of the sheer scale of data – it is clear that the capabilities provided by the current access control mechanism can be improved so that it is easier for administrators and users to address these challenges. A number of teams worked together to deliver Windows Server 2012’s Dynamic Access Control. I think you’ll find that it, like so many other things in Windows Server 2012, is just what you were asking for.If you haven’t downloaded the beta yet, take some time to read this blog and watch some of the videos it points to and then schedule some time on your calendar to download the beta and try it out.Nir Ben-Zvi, a Program Manager on the File Server team, wrote this blog.
--Cheers Jeffrey
Hello, my name is Nir Ben-Zvi and I work in the Windows Server team. I’m very excited to introduce to you the new Dynamic Access Control feature set. I’ll start with a brief introduction and insight into the planning process, discuss the new Central Access Policy model and describe the end-to-end File Server solution that we built into Windows Server 2012. I will also touch on how we enable an incremental deployment model so that you do not need to move your entire environment to Windows Server 2012 in order to use the feature set. I will touch on work with partners in this area, too.You can find a Dynamic Access Control overview demo here.
IntroductionIn today’s complex IT environments data is being created on distributed systems at a staggering rate and accessed through a plethora of devices. Compliance with regulatory standards and the need to secure leakage of business critical and personal data present major challenges for businesses and corporate IT. The key pillars for data compliance and leakage prevention are controlling who has access to information and having the ability to report who actually accessed specific information.Not surprisingly, this was the exact situation that we observed when we started planning for Windows Server 2012 a few years ago. A few of the points that we repeatedly heard from customers during the planning period included:
We then looked at the different personas within an organization and how they participate in meeting the regulatory and business requirements for data compliance, in order to provide the right set of technologies and solutions that help address the data compliance challenge. The range of personas involved starts with the CSO/CIO office that identifies the business and regulatory compliance needs. It continues with the IT administrators that manage the systems and the business owners that control the actual information. Last, the organization would like to keep the impact on the information worker to a minimum (ideally with no impact at all).
To help organizations reach their data compliance, we eventually focused on the following areas:
These focus areas were then translated to a set of Windows capabilities that enable data compliance in partner and Windows-based solutions.
Central Access PoliciesOne can look at Central Access Policies as a safety net that an organization applies across its servers. These safety net policies enhance (but do not replace) the local access policy (e.g. Discretionary ACL) that is applied to the information. For example, if a local DACL on a file allows access to a specific user but a Central Policy restricts access to the same user, the user will not be able to get access to the file (and vice versa.)The initiative to deploy and enforce a Central Access Policy may come for different reasons and from multiple levels of the organization:
Central Audit PoliciesCentral Audit Policy is a powerful tool to help maintain the security of an enterprise. One of the key goals of security audits is regulatory compliance. Industry standards such as SOX, HIPPA, PCI, etc. require organizations to follow a strict set of rules related to information security and privacy. Security audits help establish the presence (or absence) of such policies and thereby prove compliance (or non-compliance) with these standards. Additionally, security audits help detect anomalous behavior, identify and mitigate gaps in security policy and deter irresponsible behavior by creating a trail of user activity that you can use for forensic analysis.Windows Server 2012 enables administrators to author audit policies using expressions that take into account what information users are accessing and who the user is so that an organization can target audit to specific information wherever it resides. This opens the doors to richer, more targeted and easy-to-manage audit policies. It enables scenarios that until now were either impossible or too difficult to enable. For example you can now easily author audit policies such as the ones listed below:
This helps regulate the volume of audit events and limit them to only the most relevant information/users so that you can monitor access to information across multiple servers without generating an unmanageable volume of audit events. In addition, the information tagging is recorded in the audit events so that event collection mechanism can provide contextual reports such as: Who accessed all the “high impact” information in the last three months.
The File Server solutionBased on this infrastructure we built a full end-to-end Windows-based solution for Windows Server 2012 Active Directory, Windows Server 2012 File Server and Windows 8 client. This solution allows you to:
In order to support deployment across multiple file servers in the organization, we are also providing the Data Classification Toolkit that enables configuration and reporting across multiple servers. The current Beta for the Data Classification Toolkit is available for download here.
The concept of incremental deploymentOne of the core design principles of Dynamic Access Control is incremental deployments. You can start using the feature set as soon as possible to solve targeted business problems for information access and audit. You can use most of the Dynamic Access Control capabilities with the Windows Server 2012 File Server and an upgraded Active Directory domain schema. Adding a minimal number of Windows Server 2012 domain controllers will enable user claims and so on. Each part of the system that you upgrade provides you with more capabilities but it is up to you to set the pace.
Partner solutionsPartner solutions and line of business applications can further use the Windows infrastructure investments for Dynamic Access Control, providing great value for organizations that use Active Directory. A few examples of partner solutions that we have already demoed at the //build/ conference last year include:
We plan to show many additional partner integrated solutions in the upcoming TechEd US conference (Jun. 11-14, 2012) Twitter hashtag #MSTechEd
A few additional resources that you might find useful:TechNet manual (Beta): http://technet.microsoft.com/en-us/library/hh831717.aspxData Classification Toolkit (Beta): https://connect.microsoft.com/site715Hands on lab: http://technet.microsoft.com/en-us/windowsserver/hh968267.aspx (Using Dynamic Access Control to automatically and centrally secure data) Dynamic Access Control at MMS 2012: http://channel9.msdn.com/posts/Dynamic-Access-Control-Demo-and-Interview
Nir Ben-Zvi
I once talked to a doctor who told me about a recent patient that had serious medical symptoms for over a year before visiting the doctor. He said that if the patient had mentioned these symptoms when they first arose, the prognosis was very good but now the patient was in trouble. That reminded me of some advice I once heard, “Never hold anything back from your doctor”. Doctors have exactly one job: to help you. They can only help you with problems that they know about so if you aren’t completely open and honest with them, you are only hurting yourself. The other thing is that by sharing your situation with a doctor, the doctor gains knowledge and skills to help other people as well. This model and thinking applies to our Customer Experience Improvement Program (CEIP) for Windows Server 2012 Beta. That is where we ask you to allow us to collect data about the health and usage of your servers. We frequently receive questions about CEIP; ‘what is CEIP?’ and ‘how is CEIP data used?’. In this post, Karen answers these questions along with the most important question ‘why should I enable CEIP?’
Karen Albrecht, a Program Manager on the Windows Server Telemetry team, authored this post.
--Cheers! Jeffrey
When we talk to the server community about the Windows Customer Experience Improvement Program (CEIP), most people say ‘Never heard of it’. Those that have heard of it sometimes don’t enable it because they ‘don’t want to share their data’. In this blog article we will explore what CEIP is and what benefits you may receive by enabling it on your deployed Servers. We will also discuss several new features in Windows Server 2012 that make it easier to enable CEIP.
Let’s start by answering the question ‘What is CEIP?’ For those who have never seen CEIP before, using Windows Server 2012 Beta you can get there through Server Manager -> Local Server -> select the Customer Experience Improvement Program link.
CEIP is the program by which we learn how you use Windows Server 2012, in order to improve the product based on your feedback. You can join the Windows Server 2012 CEIP program in several ways. First, for pre-release beta software, such as the Windows Server 2012 Beta, CEIP is enabled by default to help us improve the software before its’ final release. Alternatively, in released products such as Windows Server 2008 R2 we provide notice through the CEIP user interface (shown above) so you can elect to opt-in to the program.
We know that you need to get the most out of your servers, especially when it comes to server performance and network bandwidth. The CEIP report collection and transfer process are light weight in order to meet this need. Windows records CEIP usage information using a high-speed tracing component, Event Tracing for Windows (ETW). ETW enables Windows Server 2012 to write out CEIP usage data no noticeable impact to server performance. CEIP usage information is transferred to Microsoft in a two part process using the Consolidator and Uploader scheduled tasks. The consolidator exports CEIP data into a compressed binary format that is ready for transfer. The binary is typically less than 1 MB in size so that the transfer has minimal impact to network bandwidth. The uploader scheduled task runs every once every 24 hours and transfers the CEIP binary data to the Microsoft frontend servers using the Windows Telemetry Protocol.
Another question we are often asked is ‘What data is collected by CEIP’? The data consists of basic information about how your server is configured and used; roles installed, features installed, settings used, and information about hardware. CEIP does not intentionally collect Personally Identifiable Information (PII). So, CEIP reports do not contain your contact information, such as your name, address, or phone number. This means CEIP will not ask you to participate in surveys or to read junk e-mail and you will not be contacted in any other way. The Microsoft Customer Experience Improvement Program privacy statement discusses, in detail, the data collected by CEIP and how we use it.
Moving on to the heart of the question, ‘What do I get for sending this data to Microsoft?’, you might be surprised in the ways Windows Server uses your data to improve the product. There are many examples beyond what is listed here. However we narrowed it down to the following to give you a flavor of some of the ways CEIP data is used to improve the product.
Diversity of Windows Certified hardware: One of the frequently asked questions we get is ‘What CEIP data does Microsoft share with partners?’ There are certain scenarios where a subset of CEIP data (but no PII) is shared with IxVs (independent hardware or software vendors) as part of hardware certification. An important part of the Windows server offering is supporting high quality drivers for a diversity of devices in market. The challenge is to understand what devices are most commonly used in market. CEIP data is used to model hardware profiles and map diversity of different devices in order to inform certification strategy for IxVs. Using this data, IxVs determine the breadth of drivers to certify (based on what is in market) and prioritize which devices get certified first (based on popularity).
Improved product experiences: CEIP data is used on a day-by-day basis to understand a broad range of feature configurations so that we can prioritize work according to your usage patterns. For example, in order to reduce the cost to setup new servers, CEIP records what settings you use. This allows us to refine default settings by tuning them to reflect most common usage patterns so it is faster for you to setup a new server. Another example of internal usage is in testing. In order to increase test coverage of real world test patterns, we analyze CEIP data to understand how the product is used. This ensures that both design and testing are driven with your usage patterns in mind. There are many, many more examples of how CEIP is used to drive customer feedback into the product but in the interest of time, let’s move on to how to configure CEIP.
After the release of Windows Server 2008 R2, we did an assessment of CEIP adoption and found that 5-7% of servers in market were reporting CEIP. While working with customers on CEIP adoption we found that although servers were opted-in we weren’t getting data from them. We did a root cause analysis and learned that the main reason servers weren’t reporting is because they are deployed in firewalled environments. To send CEIP data, servers need to be able to communicate over HTTPS (default port 443) and need to have proxy settings configured (if the server is in a network that uses a proxy server). In working with Technology Adoption Program (TAP) customers, we found that frequently one or more of these settings were not configured, thus preventing CEIP data from reaching Microsoft.
To make it easy to send CEIP data, Windows Server 2012 Beta ships several new features that allow you to get past the blocking issues so you can ‘set and forget’ CEIP. To participate in the CEIP program, the simplest way to deliver CEIP data to us is to use a new feature called Windows Feedback Forwarder (WFF). WFF is a service that proxies CEIP data from machines in a domain to Microsoft. WFF will proxy CEIP data Windows products including Windows 7 and Windows Server 2008 or higher. WFF will also proxy data for any Microsoft product that is enabled to ‘send customer feedback’.
The forwarder can sit within the domain or as an edge server. Machines in the domain are configured to send data to the forwarder via group policy. When an individual machine is triggered to collect data, it sends the data to the forwarder over HTTP and the forwarder relays the data to Microsoft over HTTPS.
An alternative method to enable CEIP is the Windows Automatic Feedback dialog, which is a new multi-machine opt-in experience that ships in Server Manager. It enables you to configure multiple individual machines to send CEIP data within just 3 clicks.
We would love to know what you think of this program and how we can improve it to provide the best experience for your deployments and Windows Server usage. Please give us your comments below.
Karen AlbrechtProgram Manager Windows Server Telemetry
The other day I was in a conversation where I drew the distinction between reliable and robust. I hadn’t really thought about it precisely but when asked to articulate the distinction I said that robust was “reliable across a wide range of conditions”. A lot of what Klaas describes in his blog about RDS reminds me of that definition. Remote Desktop Services in Windows Server 2012, is reliable across a much wider range of conditions. It works better across a wide range of networking configurations, it works better across a wide range of hardware devices and configurations (physical or virtual) and it works better across a wide range of administrative scenarios. Oh yeah, it also adds a bunch of great new features. I think you are going to enjoy what you see here.Klaas Langhout, a Director of Program Management in our RDS team, wrote this blog.--Cheers! Jeffrey
For Windows Server 2012 we listened to our customers and partners and added the most desired features and resolved the top pain points in Remote Desktop Services (RDS). Following a description of RDS, I’ll summarize some of the many dramatic improvements we have made. For those people that are not familiar with RDS, it is the workload within Windows Server that enables users to connect to virtual desktops, session-based desktops and RemoteApp programs. The key value that RDS provides is the ability to centralize and control the applications and data that employees need to perform their job from the variety of devices that the employee uses. This provides “work anywhere from any device” while ensuring that your control and compliance needs are met. In the previous release, we received consistent feedback that:
Windows Server 2012 addresses each of these issues. For Windows Server 2012 we have made RemoteFX dramatically better over a WAN as well as balancing between scale (host side cost) and reduced bandwidth. Specific improvements include:
The second main improvement area is in overall infrastructure simplification and cost reduction. Cost and complexity is a major roadblock for Virtual Desktop Infrastructure (VDI) and hosted desktop deployments of all sizes. In Windows Server 2012 we made many improvements to address this problem, such as:
The third and final focus area for improvements made in RDS has been in overall management simplification. This is targeted at improving the E2E management experience as well as enabling partner solution creation. Improvements include:
Remote Desktop Services in Windows Server 2012 provides a single infrastructure, and consistently great remoting experience even over WAN while offering three deployment choices: Session, Pooled virtual desktop collection, Personal virtual desktop collection to reduce the cost appropriate to the needs of the user. The administration is simplified and platform hooks are provided for partner extension to provide additional value and solutions.Customers are excited about RDS with Windows Server 2012 and some have already rolled out a pre-release version into production taking advantage of these new benefits! We are proud of the work we have done and look forward to providing more information as we drill into the specific features in blogs posts to come at the RDS Blog.
- The Entire Remote Desktop Virtualization Team