Follow Us on Twitter
by Richard Wilder on July 25, 2008 06:15am
I am the Associate General Counsel for Intellectual Property Policy at Microsoft, having joined the company about 9 months ago. My role is to work with a variety of constituencies inside the company and outside to help shape the approach we take to intellectual property. I am new to the company and cannot take credit for it, but am very pleased that in recent years, Microsoft has made progress in participating with open source communities. A part of that has been the implementation of the Open Specification Promise (OSP), which was launched in 2006. We think it is a simple and clear way to assure that the broadest audience of developers and customers working with commercial or open source software can implement specifications. We constantly listen to feedback from community representatives and respond to that feedback – through Q&A’s on the OSP page and directly to the community. Andy Oliver made some positive comments in this regard as recently as yesterday. When asked for clarification of the OSP with respect to the activities of Apache POI, we responded. The concerns were about implementations of specifications covered by the OSP that may be less than fully compliant – in particular due to implementation bugs. Such a situation is not explicitly covered by the OSP since it is meant to apply to a wide range of technologies and development models and it is simply not possible to address all specific situations in which it would apply. We addressed this situation in the following manner – and I apologize if the explanation is a bit technical, but I will try to avoid too much legal jargon.
The OSP says that it covers "any implementation to the extent it conforms to a Covered Specification" which addresses the heart of the conformance issue that was raised." To the extent it conforms" means that we do not require an implementation to be perfect; this can be because of implementation bugs or an intentional choice because the requirements of the particular implementation do not actually require full conformance. Under the OSP, implementations can be less than fully compliant. For example, a given implementation that takes a spreadsheet document, extracts information from it, and stores that information in a relational database might not comply with every required part of the spreadsheet document format but such an implementation would still be covered by the OSP. By way of comparison, other promises in the industry may require complete conformance for the promise to apply, and those normally require full compliance as a condition (see IBM's Interoperability Specifications Pledge). Some others make no statement about the subject at all, leaving it an open question as to whether full compliance is required. At Microsoft, we felt that unless we indicated that the OSP is more flexible, some might conservatively assume that complete compliance is required, so we included the “extent it conforms” language in the OSP. We chose to state explicitly that partially conformant implementations are covered, to the extent they are conformant in their individual implementation aspects.
As a result of this clarification, developers can have peace of mind that the specifications covered by the OSP, are, in fact, openly available without ambiguity. This is the kind of conversation and cooperation that marks our intentions with the open source community, and I look forward to continuing this dialogue into the future.
-Richard
by hjanssen on October 19, 2007 08:15pm
Yes I am still alive!
As I write this, I realize that I have been very delinquent with writing blogs. Time has gone by very fast, and I keep on working on more and more things. And before you know it, you look back and months have gone by without any blogs....
But I wanted to write a few things that have happened here in the last few months. As you might be aware, Microsoft has been working very hard on making PHP on Windows a premier experience. And working in the OSSL, my group and I find ourselves in the middle of most of these efforts. And it gives us a great view of the changes that are happening inside of Microsoft (Sometimes with a gentle shove from us )
So, we assisted the IIS team with their FastCGI effort and GoLive release a few weeks ago. (http://www.iis.net/php ) And no less interesting, something that I am particularly proud of is that Microsoft has released a CTP (Community technical Preview) of a native PHP SQL server driver on Windows.
As a result, I was invited to speak at the PHP Zend conference that was held in San Francisco last week, (www.phpcon.com) I have included the presentation for people to see what we talked about. John Bocharov from the SQL server team co-presented with me on stage and showed some of the bits that are now available for download. You can download them directly from the Microsoft download site. (here is a super long URL with the download details and helpful hints for the driver; http://www.microsoft.com/downloads/details.aspx?FamilyID=85F99A70-5DF5-4558-991F-8AEE8506833C&displaylang=en )
As part of the release, we at the OSSL converted phpbb (www.phpbb.com) to run with the native SQL server driver, and I showed phpbb running natively on SQL Server, on stage - which was pretty cool. We are currently cleaning those changes up and will be submitting them to phpbb code base. And....I am looking to see what other popular PHP applications out there we can contribute support to by writing code & integrating the driver.
All this shows a commitment from Microsoft that I am pretty excited to be a part of. I mean....an IIS 6 version of FasCGI with specific support for PHP, FastCGI build into Server 2008/IIS7 with PHP optimizations and now the SQL server driver for PHP on Windows!
What is this world coming to!
A note on the PHP driver, it is like I said - a community preview. Which means we are very much looking for your feedback on how to improve it, and any bugs you might encounter. So please participate as you can to make this the driver you would like to see it be.
Let's see...what else we have we been up to? We have been working with the Apache Software Foundation more, which has been a lot of fun. And in discussions with them to see how we could better work together, I supplied them with 75 MSDN licenses to help them develop on Windows. I am looking forward to continued cooperation with ASF on many different fronts.
The Lab continues to grow in both manpower and equipment, and we now have over 300 Servers to play with. From the old P3’s (very few still left) to the latest in IBM, DELL and HP Blade systems.
I will do my best to be more responsive and blog more. So keep the comments coming! Until the next time…………….. Thanks, Hank.
Here is great news for open source developers: Brian Harry announced today at the Microsoft’s ALM Summit that Git is now fully integrated into Visual Studio as well as the Team Foundation Service, Microsoft’s cloud-powered Application Lifecycle Management tool.
Here at Microsoft Open Technologies, Inc., we are excited to hear such news as this offers more choice and flexibility to development teams. We happen to work on a daily basis with developers on Git in the context of projects such as Node, Dash, Redis or Solr so we totally get the goodness of this news.
The Visual Studio Tools for Git work great against Git repositories locally, in Team Foundation Service, on GitHub, CodePlex, BitBucket etc. That’s all because they are using Git as the distributed source control solution and they talk to Git repositories via the open source library LibGit2. LibGit2 is a portable C library that runs on many different platforms including Linux and Mac.
Microsoft engineers in Brian’s team have been contributing to LibGit2 for a number of months now as they worked with the community to add Git support in Visual Studio – some of them earning committer rights on this popular and very active open source project. Even better as the team started testing the integration, all the bug fixes and security fixes that they found also have been contributed back to the project.
Therefore not only is Brian’s announcement good news for developers in Visual Studio wanting to use Git to contribute to open source projects, it’s also great news for others building on top of the LibGit2 library on any platform.
The Visual Studio Tools for Git are provided as an extension for Visual Studio 2012 but Brian also says that they should be included in the box with all editions of Visual Studio in a future release – including the Express editions.
I can tell you MS Open Tech engineers can’t wait to take full advantage of the Visual Studio Tools for Git in their daily interaction and collaboration with the open source developers’ community.
by MichaelF on September 11, 2006 03:16pm
Be careful what you write on your blog about having such a great time at OSCON, because the Port 25 team will find it! I mentioned that I had wanted to do a second video regarding everything I had learned at OSCON. I was (almost) embarrassed how my first video interview fell into the category of “common misconceptions about running OSS projects.” Knowing me, I had to correct this at once.
James Howison, a doctoral student on Kevin Crowston's NSF-funded research team at the Syracuse University Information School and a regular presenter at O’Reilly events, gave this incredible tutorial about OSS communities. I couldn’t wait to get back to campus to show my team what I had learned and how we were going to apply it to our power toys. So, we all decided to film the presentation in order to capture the discussions of a team at Microsoft going open and have something to share with other teams that are interested. I think this is extremely cool stuff, and I hope you agree and want to see more like it.
As a follow-up we hope to get James in for a pod cast interview in the near future to provide some feedback and insight on this topic. Stay tuned.
Links:
Presentation Outline
Presentation Slides
(This blog was originally posted on our Interoperability blog)
As a proud member of the Apache Software Foundation, it’s always great to see the growth and adoption of Apache community projects. The Apache Hadoop project is a prime example. Last year I blogged about how Microsoft was engaging with this vibrant community, Microsoft, Hadoop and Big Data. Today, I’m pleased to relay the news about increased interoperability capabilities for Apache Hadoop on the Windows Server and Windows Azure platforms and an expanded Microsoft partnership with Hortonworks.
Microsoft Technical Fellow David Campbell announced today new previews of Windows Azure HDInsight Service and Microsoft HDInsight Server, the company’s Hadoop-based solutions for Windows Azure and Windows Server.
Here’s what Dave had to say in the official news about how this partnership is simplifying big data in the enterprise.
“Big Data should provide answers for business, not complexity for IT. Providing Hadoop compatibility on Windows Server and Azure dramatically lowers the barriers to setup and deployment and enables customers to pull insights from any data, any size, on-premises or in the cloud.”
Dave also outlined how the Hortonworks partnership will give customers access to an enterprise-ready distribution of Hadoop with the newly released solutions.
And here’s what Hortonworks CEO Rob Bearden said about this expanded Microsoft collaboration.
“Hortonworks is the only provider of Apache Hadoop that ensures a 100% open source platform. Our expanded partnership with Microsoft empowers customers to build and deploy on platforms that are fully compatible with Apache Hadoop.”
An interesting part of my open source community role at MS Open Tech is meeting with customers and trying to better understand their needs for interoperable solutions. Enhancing our products with new Interop capabilities helps reduce the cost and complexity of running mixed IT environments. Today’s news helps simplify deployment of Hadoop-based solutions and allows customers to use Microsoft business intelligence tools to extract insights from big data.
The past few weeks have been very busy in our offices as we announced the creation of Microsoft Open Technologies, Inc. Now that the dust has settled it’s time for us to resume our regular cadence in releasing code, and we are happy to share with you the very first deliverable from our new company: a new and significant iteration of our work on Redis on Windows, the open-source, networked, in-memory, key-value data store.
The major improvements in this latest version involve the process of saving data on disk. Redis on Linux uses an OS feature called Fork/Copy On Write. This feature is not available on Windows, so we had to find a way to be able to mimic the same behavior without changing completely the save on disk process so as to avoid any future integration issues with the Redis code.
The version we released today implements the Copy On Write process at the application level: instead of relying on the OS we added code to Redis so that some data structures are duplicated in such a way that Redis can still serve requests from clients while saving data on disk (thus achieving the same effect of Fork/Copy On Write does automatically on Linux).
You can find the code for this new version on the new MS Open Tech repository in GitHub, which is currently the place to work on the Windows version of Redis as per guidance from Salvatore Sanfilippo, the original author of the project. We will also continue working with the community to create a solid Windows port.
We consider this not to be production ready code, but a solid code base to be shared with the community to solicit feedback: as such, while we pursue stabilization, we are keeping the older version as default/stable on the GitHub repository. To try out the new code, please go to the bksavecow branch.
In the next few weeks we plan to extensively test the code so that developers can use it for more serious testing. In the meantime, we will keep looking at the ‘save on disk’ process to find out if there are other opportunities to make the code perform even better. We will promote the bksavecow branch to master as soon as we (and you!) are confident the code is stable.
Please send your feedback, file suggestions and issues to our GitHub repository. We look forward to further iterations and to working with the Redis community at large to make the Windows experience even better.
Claudio Caldato
Principal Program Manager
Microsoft Open Technologies, Inc.
A subsidiary of Microsoft Corporation.
As Microsoft continues to deliver on its commitment to Interoperability, I have good news on the Open Source Software front: today, the OData Library for Objective-C project was submitted to the Outercurve Foundation’s Data, Languages, and Systems Interoperability gallery.
This means that OData4ObjC, the OData client for iOS, is now a full, community-supported Open Source project.
The Open Data Protocol (OData) is a web protocol for communications between client devices and RESTful web services, simplifying the building of queries and interpreting the responses from the server. It specifies how a web service can state its semantics such that a generic library can express those semantics to an application, meaning that applications do not need to be custom-written for a single source.
The Outercurve Foundation already hosts 19 OSS projects and, as Gallery Manager Spyros Sakellariadis notes in his blog post, this is the gallery’s second OData project, the first being the OData Validation project contributed last August.
“With this new assignment, we expect to involve open source community developers even more in the enhancement of seminal OData libraries,” he said.
Microsoft Senior Program Manager for OData Arlo Belshee notes in his blog post that the Open Sourcing of the OData client library for Objective C will enable first-class support of this important platform. “Combined with exiting support for Android (Odata4j, OSS and Windows Phone (in the odata-sdk by Microsoft), this release provides strong, uniform support for all major phones,” he said.
In assigning ownership of the code to the Outercurve Foundation, the project leads are opening it up for community contributions and support. “They firmly believe that the direction and quality of the project are best managed by users in the community, and are eager to develop a broad base of contributors and followers,” Belshee said.
As Microsoft continues to build and provide Interoperability solutions, Sakellariadis thanked the Open Source communities for their continued support, noting that together “we can all contribute to achieving a goal of device and cloud interoperability, of true openness.”
by Bryan Kirschner on October 14, 2008 06:57pm
It is just two months since Microsoft finalized the acquisition of Powerset, a San Francisco-based search and natural language company. Powerset's goals are to "change the way humans interact with computers through language"- improving search by indexing Web pages based on the meaning expressed in them rather than just the literal words.
Collaboration between the Powerset team and their new colleagues in Live Search has already resulted in some integration projects: Freebase Answers, improved captions for Wikipedia results, and new related searches using the Factz engine.
The application of Powerset's technology to Live Search will enable Live Search to more quickly surface the most relevant information, resulting in improvements to the end-user experience. The Powerset acquisition is an important part of Live Search's strategy, and HBase is key to Powerset's ongoing success and will also open more opportunities for other Live Search projects as well as for the broader community to move the whole web forward.
But what's especially notable is that the Powerset team has resumed contributions to HBase, an open-source, column-oriented, distributed database written in Java. The contributions relate to infrastructural storage technology enabling large scale data processing.
HBase, which is an important component of Powerset's development, is developed as part of the Apache Software Foundation's Hadoop project, and runs on top of the Hadoop Distributed File System, providing BigTable-type capabilities. (HBase initially started as a contribution to Hadoop before becoming a full sub-project of Hadoop in January 2008.)
For the past year and a half, Powerset has sponsored two full-time developers to work on HBase; Michael Stack and Jim Kellerman are also on the Hadoop Project Management Committee. Through the continued work of these developers, Microsoft will help improve HBase, which receives significant lift from the active community that supports the project.
Technology companies and communities have always collaborated (see this great research overview). There are some great examples in the past of Microsoft being a creative, agile leader - one of my favorites being the Most Valuable Professional (MVP) Program, which had its origin in organic, outside-in cooperation:
"Way back in the dark ages, Microsoft provided a great deal of technical support on CompuServe. The CompuServe FoxPro forum was extremely busy and Calvin Hsia, then an independent developer, now Developer Lead on the Fox team, created what we called "Calvin's List." It was a listing of the number of postings by person, including info on both messages sent and received. ...As the story goes, some of the Microsoft people jumped on Calvin's List as a way to identify high contributors, and thus was born the MVP program."
But if you look at how open source in particular has changed the industry from 1998 onward, as other vendors figured out ways to interact with open source, we simply haven't been the first, the fastest, or the most creative.
That history is a fact of life. But so are the implications from studying what has happened as firms and communities find ways to work together (I have a small quibble with the choice of title but not with the main point of A man on the inside: Unlocking communities as complementary assets - a "woman on the inside" would be just as effective...).
The conclusion is unambiguous: there are mutual opportunities that come from openness to working together. We're just scratching the surface on the range of opportunities for Microsoft to participate in and contribute to open source communities in ways that are good for customers, good for communities - and good for business.
The next ten years of software will also be a time of growth and change, where both open source and Microsoft communities will grow together, so it is exciting to see contribution to HBase join contribution to ADOdb, a popular data access layer for PHP used by many applications (this was Microsoft's first code contribution to PHP projects, but not the last), and OpenPegasus, an important part of System Center's new cross-platform approach.
But it is not unexpected-and others will follow.
More good news on Microsoft's commitment to Interoperability in the cloud: last week Sandy Gupta, the General Manager for Microsoft's Open Solutions Group, announced that Windows Server Hyper-V is now an officially supported hypervisor for OpenNebula.
This open source project is working on a prototype for release next month and it will soon be possible for customers to build and manage OpenNebula clouds on a Hyper-V based virtualization platform.
"Windows Server Hyper-V is an enterprise class virtualization platform that is getting rapidly and widely deployed in the industry. Given the highly heterogeneous environments in today’s data centers and clouds, we are seeing enablement of various Linux distributions including SUSE, CentOS, Red Hat, and CS2C on Windows Server Hyper-V, as well as emerging open source cloud projects like OpenStack -- and now OpenNebula," Gupta said in a blog post.
I've been involved with open source software for just shy of 15 years now. During that time I've seen open source software become a fundamental part of technology innovation. It is that technology innovation that has fed me and my family for many years. I like to think I've given back and played a small part in the growth of open source software through my code contributions and my open source strategic consultancy services. But, today I am presented with an opportunity to give back even more. From today onwards I will be joining Microsoft UK, where I’ll be serving a global role supporting the amazing team at Microsoft Open Technologies, Inc.
MS Open Tech, a wholly owned subsidiary of Microsoft, was set up to advance Microsoft's investment in openness - including interoperability, open standards and open source. Those of you who know me through my open source work will know just what this means, but for those who don't know me I guess a mini-bio might be in order.
I've held a variety of roles including software development, academic research, university lecturing and strategic consulting to both academic research community (via the OSS Watch service at the University of Oxford) and to the private sector (via OpenDirective, a small consultancy company). In all these roles open source software has been a fundamental part of my work. Through this work I've been able to contribute back to many projects, particularly within the Apache Software Foundation where I currently have the honour of standing on the Board of Directors.
More important than my history and my contributions, however, is what the open source experience has taught me. I can honestly say that I have learned far more from my open source engagements than I have in any other of my activities (including amazing experience such as being vice-captain of a national schoolboy sports team and not unsuccessful band manager). This new role is an opportunity to work in this new mixed IT world of devices and services; and play my part in maximizing Microsoft's investments in openness.
This is the first of many exciting days, I am sure. I look forward to telling you about others in the near future.
Ross Gardler
Microsoft UK (Supporting Microsoft Open Technologies, Inc.)
by jcannon on January 25, 2008 06:58pm
Abstract: The Apache authentication module mod_auth_kerb allows Apache to authenticate users against a Kerberos KDC including one from ActiveDirectory. Kerberos itself can be fairly complex to set up. This guide will attempt to show the specific steps required to make this possible as well as discuss security limitations specific to the interoperability matters. This guide assumes a basic understanding of Kerberos V and that the Active Directory domain controller is properly configured prior to starting this process.
Note: This paper represents testing and documentation in a lab environment. User Account Control (UAC) is an essential security component to Windows and Microsoft does not recommend turning off UAC in production environments.
Attachment: ApacheKerbWindowsServer.pdf
Gianugo Rabellino, Microsoft’s Senior Director for Open Source Communities, just finished delivering his keynote at OSCON in Portland. As Gianugo is now wandering around the OSCON session and expo floor, I thought it would we useful to give you a quick recap of what he just presented.
During his keynote, Gianugo discussed how both the world and Microsoft are changing, saying that “at Microsoft we continue to evolve our focus to meet the challenging needs on the industry: we are open, more open than you may think.”
Gianugo explained that the frontiers between open source, proprietary and commercial software are becoming more and more of a blur. The point is not about whether you run your IT on an Open Source stack or a commercial stack, the important thing is how you can assemble software components and build solutions on top of them using APIs, protocols and standards. And the reality is that most IT systems are using heterogeneous components, he said.
Looking at the cloud, the blur is even more opaque. What does Open Source or Commercial mean in the cloud?
Gianugo put it this way: “In the cloud, we see just a continuous, uninterrupted shade of grey, which makes me believe it's probably time to upgrade our vision gear. If we do that, we may understand that we have a challenge ahead of us, and it's a big one: we need to define the new cornerstones of openness in the cloud. And we actually gave it a shot on this very same stage one year ago, when we came up with four interoperability elements of a cloud platform: data portability, standards, ease of migration & deployment, and developer choice.”
Finally, Gianugo talked about how Microsoft’s participation in Open Source communities is real, and he used his keynote as an opportunity to announce a few new projects and updates.
Gianugo Rabellino
One way we interact with open source software is by building technical bridges, Gianugo said, giving an example on the virtualization front: announcing support for the Red Hat Enterprise Linux 6.0 and CentOS 6.0 guest operating systems on Windows Server Hyper-V (which follows this Linux Interoperability announcement at OSBC a few weeks ago. )
On the cloud development front, we are continuing to improve support for open source languages and runtimes, Gianugo said, announcing the availability of a new version of the Windows Azure SDK for PHP, an open source project which is led by Maarten Balliauw from RealDolmen, where Microsoft is providing funding and technical assistance.
Maarten has all the details on the new features and link to the open source code of the SDK. This announcement also includes a set of cloud rules for the popular PHP_CodeSniffer tool that Microsoft has developed to facilitate the transition of existing PHP applications to Windows Azure. The new set of rules is available on Github.
An on demand Webcast of Gianugo’s keynote will soon be available, and I’ll post the link to it here.
Thanks!
by MichaelF on August 13, 2007 03:10pm
I had the opportunity to present at both OSCON in Portland and at LinuxWorld in San Francisco in the last three weeks – both O’Reilly and IDG were gracious enough to grant me a session on the work that Microsoft is doing with Novell, XenSource, and others on Linux and Windows interoperability.
Overall our focus is on three critical technology areas for the next-generation datacenter: virtualization, systems management, and identity. Identity in particular spans enterprise datacenters and web user experiences, so it’s critical that everyone shares a strong commitment to cross-platform cooperation.
Here are the slides as I presented them, with some words about each to give context, but few enough to make this post readable overall. I skipped the intro slides about the Open Source Software Lab since most Port 25 readers know who we are and what we do.
Why interoperability?
The market for heterogeneous solutions is growing rapidly. One visible sign of this is virtualization, an “indicator technology,” which by its nature promotes heterogeneity. Virtualization has become one of the most important trends in the computing industry today. According to leading analysts, enterprise spending on virtualization will reach $15B worldwide by 2009, at which point more than 50% of all servers sold will include virtualization-enabled processors. Most of this investment will manifest itself on production servers running business critical workloads.
Given the ever improving x86 economics, companies are continuing to migrate off UNIX and specialty hardware down to Windows and Linux on commodity processors.
So, why now?
First, customers are insisting on support for interoperable, heterogeneous solutions. At Microsoft, we run a customer-led product business. One year ago, we established our Interoperability Executive Customer Council, a group of Global CIOs from 30 top global companies and governments – from Goldman Sachs to Aetna to NATO to the UN. On the Microsoft side, this council is run by Bob Muglia, the senior vice president of our server software and developer tools division. The purpose of this is to get consistent input on where customers need us to improve interoperability between our platforms and others – like Linux, Eclipse, and Java. They gave us clear direction: “we are picking both Windows and Linux for our datacenters, and will continue to do so. We need you to make them work better together.”
Second, MS and Novell have established a technical collaboration agreement that allows us to combine our engineering resources to address specific interoperability issues.
As part of this broader interoperability collaboration, Microsoft and Novell technical experts are architecting and testing cross-platform virtualization for Linux and Windows and developing the tools and infrastructure necessary to manage and secure these heterogeneous environments.
I am often asked, “Why is the agreement so long?” as well as “Why is the agreement so short?” The Novell-Microsoft TCA is 5 years mutual commitment. To put this in context, 5 years from now (2012) is two full releases of Windows Server and 20 Linux kernel updates (given the 2.5 month cycle we’ve seen for the last few years). This is an eternity in technology. What’s important to me is that it’s a multi-product commitment to building and improving interoperability between the flagship products of two major technology companies. This means we can build the practices to sustain great interoperable software over the long term as our industry and customer needs continue to evolve.
This talk covers two major components of the future of Linux and Windows interoperability: Virtualization and Web Services protocols.
On the Metal focuses on the virtualization interoperability work being done between Windows Server 2008 and Windows Server virtualization, and SUSE Linux Enterprise Server and Xen.
On the Wire covers the details and challenges of implementing standards specifications, such as WS-Federation and WS-Management; and how protocol interoperability will enable effective and secure virtualization deployment and management.
These are the key components required for the next-generation datacenter. We know the datacenters of today are mixtures of Windows, Linux, and Unix, x86, x64 and RISC architectures, and a range of storage and networking gear. Virtualization is required to enable server consolidation and dynamic IT; it must be cross-platform. Once applications from multiple platforms are running on a single server, they need to be managed – ideally from a single console. Finally, they must still meet the demands of security and auditability, so regardless of OS they must be accessible by the right users at the right levels of privilege. Hence, cross-platform virtualization demands cross-platform management and identity.
In non-virtualized environments, a single operating system is in direct control of the hardware. In a virtualized environment a Virtual Machine Monitor manages one or more guest operating systems that are in “virtual” control of the hardware, each independent of the other.
A hypervisor is a special implementation of a Virtual Machine Monitor. It is software that provides a level of abstraction between a system’s hardware and one or more operating systems running on the platform.
Virtualization optimizations enable better performance by taking advantage of “knowing” when an OS is a host running on HW or a guest running on a virtual machine.
Paravirtualization , as it applies to Xen and Linux, is an open API between a hypervisor and Linux and a set of optimizations that together, in keeping with the open source philosophy, encourage development of open-source hypervisors and device drivers.
Enlightenment is an API and a set of optimizations designed specifically to enhance the performance of Windows Server in a Windows virtualized environment.
Hardware manfuacturers are interested in virtualization as well. Intel and AMD have independently developed virtualization extensions to the x86 architecture. They are not directly compatible with each other, but serve largely the same functions. Either will allow a hypervisor to run an unmodified guest operating system without incurring significant performance penalties.
Intel's virtualization extension for 32-bit and 64-bit x86 architecture is named IVT (short for Intel Virtualization Technology). The 32-bit or IA-32 IVT extensions are referred to as VT-x. Intel has also published specifications for IVT for the IA-64 (Itanium) processors which are referred to as VT-i; .
AMD's virtualization extensions to the 64-bit x86 architecture is named AMD Virtualization, abbreviated AMD-V.
There are three Virtual Machine Monitor models.
A type 2 Virtual Machine Monitor runs within a host operating system. It operates at a level above the host OS and all guest environments operate at a level above that. Examples of these guest environments include the Java Virtual Machine and Microsoft’s Common Language Runtime, which runs as part of the .NET environment and is a “managed execution environment” that allows object-oriented classes to be shared among applications.
The hybrid model, shown in the middle of the diagram has been used to implement Virtual PC, Virtual Server and VMWare GSX. These rely on a host operating system that shares control of the hardware with the virtual machine monitor.
A type 1 Virtual Machine Monitor employs a hypervisor to control the hardware with all operating systems run at a level above it. Windows Server virtualization (WSv) and Xen are examples of type 1 hypervisor implementations.
Development of Xen and the Linux hypervisor API paravirt_ops began prior to release of Intel and AMD’s virtualized hardware and were designed, in part, to solve the problems inherent in running a virtualized environment on non-virtualization-assisted hardware. They continue to support both virtualization-assisted and non-virtualization-assisted hardware. These approaches are distinct from KVM, or the Kernel-based Virtual Machine, supports only virtualization-assisted hardware; this approach uses the Linux kernel as the hypervisor and QEMU to set up virtual environments for Linux guest OS partitions.
In keeping with the open source community’s philosophy of encouraging development of open source code, the paravirt_ops API is designed to support open-source hypervisors. Earlier this year VMware’s VMI was added to the kernel as was Xen. Paravirt_ops is in effect a function table that enables different hypervisors – Xen, VMware, WSv – to provide implementation of a standard hypercall interface, including a default set of functions that write to the hardware normally.
Windows Server 2008 enlightenments have been designed to allow WS 2008 to run in either a virtualized or non-virtualized environment *unmodified*. WS 2008 recognizes when it is running as a guest on top of WSv and dynamically applies the enlightenment optimizations in such instances.
In addition to a hypercall interface and a synthethic device model, memory management and the WS 2008 scheduler are designed with optimizations for when the OS runs as a virtual machine.
The WSv architecture is designed so that a parent partition provides services to the child partitions that run as guests in the virtual environment. From left to right:
Native WSv Components:
Interoperability Components:
Like the WSv architecture, the Xen architecture is designed so that a special partition, in this case Dom 0, provides services to guest partitions that run in a virtual environment.
Native Xen Components:
The slide says it all… I couldn’t figure out a way to put this one in a graphic. ;)
Virtualization interoperability testing is very challenging. While the architecture may look similar at a high level, the devil is in the details – down at the API and ABI level, the technologies are quite different.
From a personnel standpoint, the expertise required to debug OS kernels is hard to find, let alone software engineers with these skills who are focused on writing test code. Microsoft has established a role known as “Software Design Engineer in Test” or “SDE/T” which describes the combination of skills and attitude required to test large-scale complex software rigorously through automated white-box test development.
The problem of testing Linux and Windows OSes across WSv and Xen requires these kernel-level skills, but on both operating systems. It’s a non-trivial challenge.
Next is the technical issue of the test matrix:
To put this in context, we need a minimum of 40 server chassis to test this matrix – for each operating system.
On top of this, the software components that must be tested include:
Since Windows and Linux are general-purpose operating systems, these components must be tested across a range of workloads which will guarantee consistent, high-performance operation regardless of usage (file serving, web serving, compute-intensive operations, networking, etc.).
Finally – and no less a challenge than the skills and technology aspects – is that of building a shared culture between two very different and mature engineering culture. What is the definition of a “Severity 1” or “Priority 1” designation for a defect? How do these defects compete for the core product engineering teams’ attention? How are defects tracked, escalated, processed, and closed across two different test organizations’ software tools? Most importantly, what is the quality of the professional relationships between engineers and engineering management of the two organizations? These are the critical issues to make the work happen at high quality and with consistency over the long term.
WS-Management is an industry standard protocol managed by the DMTF (Distributed Management Taskforce), whose working group members include HP, IBM, Sun, BEA, CA, Intel, and Microsoft among others. The purpose is to bring a unified cross-platform management backplane to the industry, enabling customers to implement heterogeneous datacenters without having separate management systems for each platform.
All Microsoft server products ship with extensive instrumentation, known as WMI. A great way to see the breadth of this management surface is to download Hyperic (an open source management tool) and attach it to a Windows server – all of the different events and instrumentation will show up in the interface, typically several screen pages long.
It is not surprising that the management tools vendors are collaborating on this work – and it’s essential to have not just hardware, OS, and management providers but application layer vendors like BEA as well – but to me the most important aspect of the work is the open source interoperability.
In the Microsoft-Novell Joint Interoperability Lab, we are testing the Microsoft implementation of WS-Management (WinRM) against the openwsman and wiseman open source stacks. This matters because the availability of proven, interoperable open source implementations will make it relatively easy for all types of providers of both management software and managed endpoints to adopt a technology that works together with existing systems out of the box. Regardless of development or licensing model, commercial and community software will be able to connect and be well-managed in customer environments.
So what does this all mean? We’ll see end-to-end interoperability, where any compliant console can manage any conforming infrastructure – and since the specification and the code are open, the barriers to entry are very low. It’s important that this capability extends to virtualized environments (which is non-trivial) so that customers can get the full potential of the benefits of virtualization – not just reducing servers at the cost of increased management effort.
Sometimes people challenge me with the statement “if you would just build software to the specification, you wouldn’t need to all this interoperability engineering!” This is in fact a mistaken understanding of interoperability engineering. Once you’ve read through a specification – tens to hundreds of pages of technical detail – and written an implementation that matches the specification, then the real work begins. Real-world interoperability is not about matching what’s on paper, but what’s on the wire. This is why it’s essential to have dedicated engineering, comprehensive automated testing, and multiple products and projects working together. A good example of this is the engineering process for Microsoft’s Web Services stack. The specifications (all 36 of them) are open, and licensed under the OSP (Open Specification Promise). In the engineering process, Microsoft tests the Windows Web Services implementation against the IBM and the Apache Axis implementations according to the WS-I Basic profile. A successful pass against all these tests is “ship criteria” for Microsoft, meaning we won’t ship our implementation unless it passes.
In the messy world of systems management, where multiple generations of technologies at a wide range of ontological levels (devices, motherboards, networking gear, operating systems, databases, middleware, applications, event aggregators, and so on) testing is complex. Adding virtualization into this mix adds another layer of complexity, necessitating methodical and disciplined testing.
Open ID is a distributed single sign-on system, primarily for websites. It’s supported by a range of technology providers including AOL, LiveJournal, and Microsoft.
WS-Federation is the identity federation web services standard which allows different identity providers to work together to exchange or negotiate information about user identity. It is layered on top of other Web Services specifications including WS-Trust, WS-Security, and WS-SecurityPolicy – many of which are lacking an open source implementation today.
ADFS is Active Directory Federation Services, a mechanism for identity federation built into Microsoft Active Directory.
Cardspace is an identity metasystem, used to secure user information and unify identity management across any internet site.
Project Higgins is an Eclipse project intended to develop open source implementations of the WS-Federation protocol stack as well as other identity technologies including OpenID and SAML.
Samba is a Linux/Unix implementation of Microsoft’s SMB/CIFS protocols for file sharing and access control information. It is widely deployed in Linux-based appliances and devices, and ships in every popular distribution of Linux as well as with Apple’s OS X.
This work is still in early phases, and you can expect more details here in the future. Mike Milinkovich of Eclipse has been a champion for improving the interoperability of Eclipse and Microsoft technologies, especially Higgins. Separately the Bandit Project has made significant progress in building technologies which support CardSpace. I appreciate the work of these teams and look forward to more progress here.
The slide says it all here. We’re committed to long term development and delivery of customer-grade interoperability solutions for Windows and Linux, and we’ll do it in a transparent manner. Tom Hanrahan, the Director of the Microsoft-Novell Joint Interoperability Lab, brings many years of experience in running projects where the open source community is a primary participant. I and my colleagues at Microsoft are excited to learn from him as he puts his experiences at the OSDL/Linux Foundation and at IBM’s Linux Technology Center into practice guiding the work of the lab.
You can expect regular updates from us on the progress and plans for our technical work, and I expect you to hold me and Tom accountable for this promise.
I hope you found the presentation valuable. I felt it was important to get this material out broadly since it will impact many people and essential to be clear about what we are building together with Novell, XenSource, and the open source community.
I spent the entire week last week enjoying some good weather and southern hospitality in the Carolinas. On Tuesday Mar 15 I had the pleasure of being invited to present at the Charlotte Enterprise Developers Guild organized by Bill Jones (special thanks to SystemTec for sponsoring the evening). On arrival I found the best dressed group of developers I have seen in a long time and felt compelled to apologize for my jeans and Converse. They sure keep it classy down South – something us Northwesterners could probably stand to learn from.
The focus of the talk (and subsequent discussion) was Java and PHP on Windows Azure. I was pleased to learn that the group consisted of a healthy mix of developers writing Java code, PHP code, and .NET code. In fact, close to 50% of the folks in the room indicated that they use multiple runtimes in their local data centers and are used to interacting with multiple codebases consisting of different languages. We had a great conversation about what it means to move to the Cloud and the approach Microsoft is taking to building an open an interoperable platform that will provide a robust general purpose platform for languages and runtimes far beyond .NET. I got a lot of great feedback on the Eclipse and ANT tooling that was recently announced and have opened some new discussions on additional work we are exploring enable additional Java developer workflows / build systems including Apache Maven. Stay tuned on this!
I also had the pleasure of attending the 4th Annual POSSCON (Palmetto Open Source Conference) in Columbia, South Carolina where Microsoft was a sponsor. The speaker lineup was great and there were a number of interesting sessions on the agenda particularly related to open source in mobile applications which seemed to be the hot topic of the event. I was pleased to finally meet such OSS icons as Jim Jagielski of the Apache Software Foundation, and attend a number of great sessions by other well known OSS advocates including William “whurley” Hurley, Bob Sutor of IBM and Jon “maddog” Hall.
My colleague Gianugo Rabellino had the opportunity to present a keynote at the event and took the opportunity to showcase a lot of open source work that is underway both in Microsoft and the Windows ecosystem. He described the change underway in Microsoft toward greater openness and discussed the future of collaboration between Microsoft and the the many Open Source communities on objectives we all share as technologists.
My best booth award (from a coolness factor point of view) goes to my new friends at RepRap.org who are working on building self replicating open source 3D printers. This is a sweet mashup of open source software, open hardware design, commodity component architecture and pure geekitude. We had a great debate about the future of self replication and when we were done my head hurt but it was a blast.
The conference had around 500 attendees and I even somehow became the mayor on Foursquare despite only checking in twice. It was a great time and I look forward to seeing what is in store next year at POSSCON 2012.
Craig Kitterman Sr. Technical Ambassador @craigkitterman http://craig.kitterman.net
As you know, educational institutions across the globe face the challenge of trying to meet accelerating technology demands with limited budgets. That is why I am really pleased to report that, today, Moodlerooms has released an open source plug-in that integrates Microsoft Live@edu with Moodle, the world's leading open source Learning Management System with 55 million users.
With this integration, Moodlerooms will make the rich functionality of Live@edu directly accessible within the Moodle 2.0 and 1.9 environments via single sign-on, further enabling teachers and students to access the quality, enterprise-level tools they need to effectively teach and learn online.
Microsoft Live@edu makes hosted email, communications and collaboration services freely available to educational institutions and currently benefits more than 22 million people worldwide.
Moodlerooms’ Microsoft Live@edu services plug-in for Moodle will bring Microsoft’s Live@edu email, calendar, instant messaging and Bing services right into the Moodle classroom environment.
The plug-in is freely available to download from the moodle.org plug-in directory here.
You can read more about this on the Moodlerooms release here, as well as on the Microsoft in Education blog and the Microsoft Openness blog.