Today is a very important day in the history of Microsoft; it's a moment with which I am very happy to be associated. If you didn't see the big news, you can read about it here. But if you just want to skip to the "What does this mean for Office and File Formats?" part, read on.

To recap what was announced, Microsoft is making changes to its technology and business practices in the area of interoperability. These advancements are designed to make our products more open and more available to the broader software community. There are four central principles involved:

• Ensuring open connections to our high-volume products
• Promoting data portability via open document formats, interfaces, import/export functions and other means
• Enhancing support for industry standards
• Engaging more actively (than we currently do) with the industry at large on standards and other interoperability issues

This is consistent with a path we've been on with Office for quite some time. This is also reflected by many actions we've taken over the past 5 years. By "change," one can really argue that we're moving out of the slow lane into the fast lane on this topic, but we're still on the same road traveling in the same direction. A great example of the steps we've been taking is our announcement last week of broad, public availability of the Binary file format documentation.

To recap some of the ground we've already covered with respect to interoperability, here's a short list of background reading that is helpful: Interoperability Executive Customer (IEC) Council, Interoperability Vendor Alliance (IVA), CodePlex, Interoperability Home, OpenXMLDeveloper.org, OpenXMLCommunity.org, Accessibility, Interop Agreements with Novell, Xandros, Linspire, Turbolinux, IdeaAlliance & XML 2007 Conference, Virtualization, OpenID, JVC, Health Care, ADO.NET, the OSP, Microsoft Public License (MPL), Microsoft Reciprocal License (Ms-RL) and a host of others (hopefully the point is clear).

There's a specific aspect of this announcement that I wanted to highlight, because it is very relevant to the ODF discussion (as well as other file formats.) You may have seen this text in connection with the announcement:

"Enhancing Office 2007 to provide greater flexibility of document formats. To promote user choice among document formats, Microsoft will design new APIs for the Word, Excel and PowerPoint applications in Office 2007 to enable developers to plug in additional document formats and to enable users to set these formats as their default for saving documents."

For Office 2007, we will design and make available interfaces that can enable different file formats to be set as a default format in Word, Excel and PowerPoint. This addresses a key concern of organizations like BECTA who desire to have ODF or UOF or other formats enabled as a "default" for Microsoft Office applications. Should other file formats emerge in the future that are suited to word processing, spreadsheets or presentations, these new interfaces will enable their use with Office as well. In the past few years, we've seen a lot of momentum around document format support, and today's announcement will only improve the ability for others to integrate with our products.

It's important to note that the interoperability principles are unrelated to the current ISO standardization process for Open XML, and that process will proceed without regard to what is being announced today. But the objectives of Open XML now have an increased focus and sharpness. The purpose of the Ecma standard (and proposed ISO standard) is to represent content that exists in billions of binary documents, as well as delivering the type of business process integration enabled by the use of custom-defined schema. Open XML is unique in this regard.

We've said this before, but the goals of Open XML are distinctly different than ODF, PDF or UOF, and hopefully we can begin to separate the conversation about product functionality from the necessity for the Open XML standard. In our view, these have always been different conversations. The addition of these interfaces removes a potential obstacle to the adoption of other standards within our products.

The change should also underscore the idea that support for Open XML is not the same as opposition to other standards, despite many claims to this effect. Different formats are a means to achieving specific types of work, and interested communities exist to offer support for them. Today's commitment creates new opportunities to use many document formats in Office, and will allow people a greater ability to choose the formats that best suit their specific needs. This is good for our customers, but it's good for our business, too; adding these interfaces makes a lot of sense.

I saw Bob Sutor's post last week titled "There is humor in the OOXML morass." This is where he calls out comments from Nick Tsilas as worthy of "a good laugh." I interpreted Bob's post as an attempt by IBM to deny that they are leading the charge against Open XML.

I like a good laugh too, so I had a look. I looked through the history of Rob Weir's blog, just to see what the chair of the OASIS Technical Committee is doing to drive the agenda for ODF, and to see if there was any sense of balance between the IBM Open XML agenda and the IBM ODF agenda. The only thing "funny" to me here is the revelation of the data behind the (Nick's) claim. So this blog is again an attempt to sort out what IBM is really saying / doing. Confusion reigns supreme.

I took somewhat of an informal look at Rob's blogging history, and it really does illustrate IBM's tactics on Open XML. I didn't use his tagging, I just read them and offered my own thoughts. It is quite clear that at least part of IBM is campaigning against Open XML. I'm not sure why Bob wants to hide from that.

By my count, Rob Weir has 134 blog posts in his archive.

94 of those posts have a central anti-Open XML and/or anti-Microsoft theme.
24 of the posts are about ODF and related technology, momentum, OASIS news, etc.
Many of the Open XML posts are anti-Open XML posts with significant ODF discussion
Others have a focus that is dedicated to PDF, gardening or other things.

Even in the best possible light, we have a 3:1 slant toward opposing Open XML instead of touting ODF. This anti-Open XML stream originates from the co-chair of the OASIS ODF Technical Committee? Seems like Rob's attention is somewhat diverted. Having read most of the posts, some contradictions are apparent.

I have more to do than criticize Rob Weir, so I'll have to stop here. Between the data and the examples, though, the problem is clear. I really view this IBM-centric position as self-defense against reality. It seems like if the chair of the ODF TC wanted to improve adoption of ODF, then the blog would reflect that.

I wonder if the paranoia here is because of the contrast represented in the uptake of the Compatibility Pack vs. the ODF Translator for Word? The current count is 20 Million for Open XML vs. under 237,000 for the ODF Translator… or perhaps this list: http://www.openxmlcommunity.org/applications.aspx, which is A LOT longer than this one: http://opendocument.xml.org/products. (It's also worth noting that nearly every product mentioned on the OpenDocument site also supports Open XML.)

Facts are facts here. If you strip away the adjectives and adverbs, we can have a discussion about reality, based on data. Step 1, however, is acknowledging that out sound-biting each other won't get us to a situation where we can achieve interoperability with document formats. Effort and cooperation are required. Productive discussion should come to the fore. It's hard to sustain a relevant dialog with the industry when the signals from its constituents are mixed up like this.

I've included my swag at the Rob Weir posts.

Microsoft has been and continues to be fully committed to opening its document formats for Word, Excel and PowerPoint. Interoperability is not new to Office, and Open XML is part of a much broader strategy around interoperability for Office. When we look at the past three years of document format related investments, you'll see this shining through; we've done quite a lot. Different circumstances led to each of these activities, but as a collection of work, the intent is unmistakable, and despite claims to the contrary, we're highly motivated to ensure that we can participate in an open environment. These are all steps toward openness, which is good for us, good for our customers and good for the industry.

Brian Jones has covered the history of the formats and XML support for Office in a prior post, there is a significant amount of ground covered in his post. 

Let's take a look at what has happened:

• Binary File Format Program – It's amazing that many Open XML opponents today claim that the binary formats for Word, Excel and PowerPoint, "aren't available." The first license grant that I can track happened in 2003. Among the companies who have licensed the formats are Sun, Adobe and IBM. If you're interested in getting these specifications, go here: http://support.microsoft.com/kb/840817/en-us
  
  
• Covenant not to Sue for XML reference schemas - http://office.microsoft.com/en-us/products/HA102134631033.aspx?pid=CL100796341033. The covenant was issued to allay IP concerns regarding the adoption of the 2003 XML reference schemas for WordProcessingML and SpreadsheetML. The covenant makes any Microsoft patents required to implement these schemas available to anyone to implement them – and no fee is charged.  The Covenant was extended to Open XML when it was submitted to Ecma for standardization.
  
  
• Open XML Announced at TechEd 2005 – After much deliberation with analysts and customers, Microsoft announced the transition away from binary and the inclusion of Open XML in June of 2005, at the TechEd conference (this was one of the first major activities I completed at Microsoft). We treated this very carefully, as we knew from the Office '97 launch that changing file formats in Office is a hard thing to do. The transition was important for a lot of reasons. Open XML offers a lot of new opportunities for the Office ecosystem, and it helps our customers and partners get better interoperability with the document formats. We hash and re-hash the benefits a lot, but when you boil it down to its essence, this is what we're really after.
  
  
• PDF for Office 2007 (including PDF/A) – In October of 2005, Microsoft announced support for saving PDF documents within Office, including files that support the ISO standard PDF/A. If you've never encountered PDF/A, it was among the first ISO-ratified PDF standards (variations of PDF/X were first) intended to improve document archival.
  
  
• Submission of Open XML to Ecma International and ISO – In November of 2005, Microsoft announced the submission of the Open XML formats to Ecma International for Standardization.
  
  
• Open XML Developer.org, Open XML Community.org – In support of Open XML adoption, we elected to offer a developer portal that was outside our traditional MSDN developer channel. Open XML Developer is a portal for the Open XML community to share information. It is outside of MSDN because some community members would prefer to conduct their business somewhere other than Microsoft.com. Open XML Community.org is another web property that is designed to illustrate the overwhelmingly strong traction the formats have received in a short period of time. Many Open XML adopters have taken the time to add contributions to the site.
  
  
• Office compatibility Pack for Open XML formats – In November 2006, Microsoft released the first version of the compatibility pack for Microsoft Office. This add-in allows users of Office 2000, XP and 2003 to read and write Open XML formats for free. We created this add-in to retrofit Open XML support into prior releases, to ease the pain of transitioning file formats that we experienced in the Office 97 change. This add-in has been downloaded more than 20 million times, and is in widespread use today.
  
  
• Office File Scanning & Conversion tools – Early in 2007 came the release of the Office Migration Planning Manager for Office 2007, we introduced two very important tools for managing a transition to XML-based formats: The Office File Scanning tool and Office File Conversion tool. The file scanning tool scans machines for the presence of different file formats, examining for age, presence of DRM & Macros and other criteria. The reports generated by the file scanning tool can be used as the basis for the Office File Conversion tool, which converts Word, Excel and PPT binary to Open XML.
  
  
• Open Specification Promise for Open XML formats – Based on feedback we got about the CNS and because of the growing significance of interoperability, we introduced a new Open Specification Promise to give implementers of our published specifications more reasons not to worry about any Microsoft patents required to implement the Open XML specs. Open XML is offered under this promise, to ensure there is no ambiguity in our commitment to allowing others to implement the spec. My favorite phrase in the OSP (that seems to have been 'accidentally' over looked by most Open XML opponents): "In connection with the specifications listed below, this Promise also applies to the required elements of optional portions of such specifications."
  
  
• ODF Translator – The growing relevance of the ISO standard ODF was an opportunity for us to improve our interoperability approach in Office. But ODF didn't come without its challenges. Adding ODF support to Microsoft Office was always going to be tricky; we would either be accused of "embrace and extend" for adding custom extensions for all the stuff that was missing from the spec (spreadsheet formulas or presentation tables, for example), or we would build a translator that didn't move everything from Open XML to ODF, again setting us up for an accusation of hindering ODF. We took the decision to conduct that activity in an open source environment. This allowed us to accomplish a few things: First, we could sponsor the implementation of ODF in a public forum to allow for public examination / comment BEFORE the tool shipped. If anything was not being implemented satisfactorily, it could be noted and corrected. The OSS effort for the ODF Translator also allowed us to sponsor an OSS project that facilitated document translation between Open XML and something else. Early on in the life span of Open XML, this was an important activity to jump-start adoption. To date, the ODF translator is not in widespread use (the ODF Translator for Word has been downloaded ~225,000 times compared to the 20 Million for the Open XML compatibility pack), but we'd expect that the translator activity to be commensurate with interest in ODF in our customer base.
  
  
• UOF Translator – Much like the ODF Translator (and DAISY translator below), we sponsored the Open XML to UOF translator in an OSS environment because we wanted to ensure we had appropriate industry cooperation in implementing the specification.
  
  
• Microsoft Office Isolated Conversion Environment (MOICE) – MOICE is a tool based on the compatibility pack for Office, and is designed to facilitate document conversions to Open XML in a shielded process. This will protect a system against malformed binary documents and silently convert them to Open XML before they are opened in an application. The tool is available for IT administrators to deploy in high-security environments, and is available as a manual download from Microsoft.com (READ: Conspiracy theorists, we don't push that download to anyone's machine, it's a manual, opt-in download, just like the compatibility pack.)
  
  
• DAISY Translator – Of all the software projects I've ever been involved with, this is one that I get pretty warm and fuzzy about. Working at a company this size, we tend to look at macro-scale events and enablement. But with the DAISY Translator work that we've undertaken with the DAISY Consortium, we're reaching out directly to a specific community of users, and providing functionality for Office that they've wanted for a very, very long time. This work directly benefits people; this isn't about politics, positioning or anything else. This is pure up-side for the users of our products, and it is something that a lot of people feel very, very good about.
  
  
• Binary document conversion – Microsoft recently announced its intent to sponsor an Open-Source software development project to enable the conversion of binary documents to Open XML documents, making it even easier to migrate off the old binaries. At the same time we also announced that the existing binary format program will be available on the web and that Microsoft binary formats will be offered under the Open Specification Promise.
  

So, no matter how you look at it, or which nits you'd want to pick (correct or not), this is a long list of advancements that really illustrate how much we have moved forward on openness and interoperability with respect to document formats in the Office products. Hopefully I don't have to write a grand conclusion, the data here should speak for itself. Let's just say that the commitment to openness here is evident and unquestionable.

I was pointed at a document created by the ODF alliance (with a creation date of Feb 6^th) that discusses the recent dust-up on supported file types in Office 2003 Service Pack 3. While this was not one of our shining moments as a product, I was disappointed to see the amount of gross factual error in the report.

I'd like to spend some time illustrating how the report is wrong, and how the ODF Alliance ignored available data in preparing it. By doing so, of course, I hope to illustrate the level of credibility with which such claims should be regarded. All the documents / information I reference were available long before the publishing date of this material.

A Knowledge Base article discusses the actual changes made: http://support.microsoft.com/kb/938810

Patrick Durusau, Editor of the ODF specification(s) has offered another post in the vein of "can't we all just get along?" For all the discussion however, these documents from Mr. Durusau indicate a good path forward. This path is similar (the same) as what has been undertaken by DIN a while back. This seems to be a pretty reasonable approach; let's convene in an independent forum and discuss the real differences between Open XML and ODF (not just blinking text and spreadsheet formulas), and find out what interoperability really means.

Fundamentally this is a question of "Unification" vs. "Harmonization." My take on the issue is that people use these terms interchangeably, but I do not think this is the right way to conduct the discussion. "Unification" results in a THIRD format, not one single format. The Harmonization path (identifying the differences and finding ways to handle those) seems to be the area of convergence on the topic. I am a believer in this approach.

Rob Weir seems to have a different approach; it will be interesting to see who prevails within the ODF TC. While Rob and others may love to pull my quote about ODF functionality in our products, let me be clear: I do not believe it is feasible just to add features of one format to another. These formats are not subsets and supersets of each other, there are fundamental differences in text, table, graphic and style models, spreadsheets have a very different representation, and on and on and on. "Unification" points toward an argument about how product code bases will have to be re-written, and there are no winners in that discussion.

Suggesting that one can just copy / paste between these formats because they appear to be "90% similar" is an insincere / inadequate / uninformed attempt at understanding the issues that are involved.

Given my remarks about the current chair of the OASIS ODF TC, I thought I'd round out my ODF related discussion with something a bit more positive. In fact, I don't even have to add any commentary of my own, I'd just like the work to speak for itself. Patrick Durusau, editor of the ISO26300 and OASIS ODF specification, has offered some great comments about Open XML. As I've already discussed, we've made several steps toward openness with Open XML, it's great to see this recognition from someone who has seen a lot of the progress on Open XML.

http://www.durusau.net/publications/OpenXMLPosterChild.pdf

Rob Weir is apparently offended that someone would suggest that ODF is controlled by a single vendor, going so far as to accuse Burton Group of some influence over the Open XML ISO voting process. He says:

"Waiting until after February, after the DIS 29500 process concludes, to make corrections is unacceptable. Since your stated purpose in making this report public was to "advance the debate" in the current OOXML ISO process, withholding factual corrections until after that process concludes would imply that you and the Burton Group see no problems with knowingly persisting in influencing an ISO ballot with false information published under the Burton Group name. I don't believe that is the image that the Burton Group would want to project. So I urge that a correction is in order now."

http://www.robweir.com/blog/2008/02/punct-contrapunct.html

This is feedback to three posts from the Burton Group that are responses to the ODF Alliance.

http://ccsblog.burtongroup.com/collaboration_and_content/2008/02/burton-groups-r.html
http://ccsblog.burtongroup.com/collaboration_and_content/2008/02/burton-groups-1.html
http://ccsblog.burtongroup.com/collaboration_and_content/2008/02/burton-groups-2.html

But as we see with Rob over and over again, he seems to be on both sides of the argument. Here's Rob throwing those same sticks and stones at ECMA TC-45.

"This should sound familiar. OOXML is nothing more than the preferences of Microsoft Office. Whenever Word changes, OOXML will change. And if you are a user or competitor of Word, you will be the last one to hear about these changes. ISO does not own OOXML. Ecma does not own OOXML. OOXML, in practice, is controlled and determined solely by the Office product teams at Microsoft. No one else matters." http://www.robweir.com/blog/2007/12/those-who-forget-santayana.html

"So what Ecma is offering SC34 is nothing close to what was promised. Ecma is really seeking to transfer to SC34 the responsibility of spending the next 3 years fixing errors in OOXML 1.0, while future versions of OOXML ("technical revisions") are controlled by Microsoft, in Ecma, in a process without transparency, and as should now be obvious to all, without sufficient quality controls."
http://www.robweir.com/blog/2007/12/bait-and-switch.html

"First, Microsoft has managed to get JTC1 to clamp down on information. What was a transparent process is now mired in multiple levels of security leading to delay, denial of information to some NB participants and total opaqueness to the public."
http://www.robweir.com/blog/2007/12/662-resolutions-but-only-if-you-can.html

These three (along with a lot of other) statements are of course inaccurate. So if we're recommending retracting things for the sake of accuracy, it might be appropriate to start here: http://www.robweir.com/.

Yesterday was a big day, filled with lots of goodness on openness and interoperability. Today I thought I'd share some stuff that is a little different than my normal focus, but very interesting nonetheless. I realized last night (thanks to a not-so-gentle reminder from my wife) that I haven't gone through a 24-hour period without saying "XML" for about 2 years. (at one point I think I went through a similar span on the topics of color depth, tone ranges and CCD conversions for digital imaging, but that was before I had the opportunity to spill it all out into a blog post.) No matter how many books I read about the Universe, it seems to occupy a lot of my offline discussion time.

MSDN has published a new interactive developer map for the 2007 Office system. This is an innovative way to get your head around what we're offering for integration potential today.

I also shoot a lot of photographs. These are some fond memories of my Open XML days, from Seattle (with some Open XML partners on a photo shoot), Thailand and India. With apologies for the personal disclosure, I've spent a lot of my XML thinking time this week considering just how far we've come in the past few years.

And as I head out for the weekend, I thought I'd point at a blog post I saw on ZDNet that I found interesting. I do appreciate it when folks take the same step back that I do and vote for cutting us some 'slack.' We are doing a lot. We want to do the right things. So I'm happy to end my week on an upbeat: http://blogs.zdnet.com/Howlett/?p=322

This week Mike Walker announced a new "OBA Component Library" for Financial Services (OBA == Office Business Applications). If you're not familiar with OBAs, they are applications that are designed to help you use Office to get more out of the data center. We see countless situations where our customers have made a tremendous investment in line-of-business applications, but have failed to utilize the data these systems capture and manage. OBAs are designed to help solve this problem. By combining the familiar Office environment with (standards-based) integration capability, OBAs are quite useful in helping automate business processes and getting data in front of people so they can actually do something with it.

The OBA Component Library for Financial Services is a great example of how much interoperability is baked into Office, and illustrates pretty clearly how integration is fundamental to our success. Take a look at what is included in the library to get a sense for what these guys are up to:

What is included?

1. Site Templates
2. Master Pages
3. Silverlight Components
4. Web Parts
   1. BDC Web Parts
   2. Extending Enterprise Content Management Web Parts
5. Industry Standards based Infopath Forms
6. Document Generation Framework
7. Custom XML based OpenXML Documents (Excel/Word)
8. State Machine WF Workflows
9. Series of standards based integration BDC Entities
10. WS-* enabled ACORD Web Services Messages
11. Office Communications Server Components

Note the inclusion of Open XML… one piece of many related to interoperability of Office. We get hung up on the file format related parts of the interop discussion, and forget that there is an entire platform behind the interop efforts. OBAs are a great illustration of that breadth, and underscore the commitment that Office and Microsoft have made to it.

Keep an eye out for the launch here: http://msdn.microsoft.com/FinServArch

http://interopvendoralliance.org/labs/

The Interop Vendor Alliance has been testing various scenarios with Open XML interoperability. If you're not familiar with the Interop Vendor Alliance (IVA), you can read about their activities here. They've done some interesting work in the area of identity management, system management and other things. Their testing of Open XML involves translation to ODF, use of custom-defined schema and mobility scenarios where Open XML documents are being utilized on mobile devices.