I can't help but observe the "discussion" underway with respect to spreadsheet interoperability that Rob Weir has started. Essentially Rob is complaining that Microsoft didn't implement the formula namespace of OpenOffice.
For the chair of the committee to post vitriol like this about the implementation of his own format raises a number of very concerning problems.
I'd like everyone reading the post to know that Rob was invited to participate in the DII events leading up to the SP2 release, and offered the opportunity to test the beta software specifically for the purpose of providing feedback on the implementation. Normally the chair of group of the standard being implemented would jump at the chance. Rob didn't, electing instead to wait for the shipping version and then claim that it is somehow deficient to other ODF implementations that he has deemed suitable for his purposes.
Does it make sense to have a chair for the ODF TC whose apparent mission is to create a caste system for ODF implementers? Do we really think Rob, who debates whether the tough (and publicly vetted) implementation decisions of his constituents are "malice" or "incompetence?" – is this the hallmark of a leader in the standards community striving for innovation using open technologies? Is this the characteristic that OASIS wants to promote in the development of technology standards? In Rob, do we really have a person capable of operating in a vendor-neutral forum? If departments within 18 various governments really do use ODF as their standard, should we be comfortable with an ODF TC chair that is trying very hard to discredit and divide its supporters?
Is it time for Rob to step down as chair? I think so.
I'm not saying Microsoft (or anyone) should be the chair instead, but I am saying that Rob is unfit as a leader given his inability to separate his personal venom from his role as a leader in driving the standard forward. It seems like a better approach to empower people on the ODF TC who have a long-term view of the need to enable interoperability, and to move those with more short-term vendor-oriented agendas to the side.
John Head is on point with this post. eWeek seems to be fine with SP2.
As far as I can see, the only thing that Rob is really demonstrating here is that the "grossly inadequate" formula support of ODF (those are the words of David Wheeler, leader of OpenFormula, read on for details) is causing problems with vendors implementing the standard. He instead resorts to scoring implementations based on a percentage of common ground, rather than conformance to something written on paper. This gives Rob the freedom he needs to define his own criteria for what ODF implementation is, and who is doing it according to his rules.
Rob seems to be positioning himself as the final arbiter on what is "good" ODF vs. "bad" ODF. OASIS? specification? – Unimportant when Rob Weir can arbitrarily define criteria for what he thinks is good. He's in a position where only he will declare his own ODF preferences as the blessed implementation. It seems that neither the ODF TC nor the spec matter anymore. It seems that ODF is being run by an individual.
Current ODF standards do not support formulas no matter how much Rob wishes it to be so. Implementations of ODF spreadsheets are application-dependent. ODF 1.2 is not an approved standard. OpenFormula is not an approved standard. While it may be that both are on a path to standardization in the future, today they are not. This is a situation that has been known to the ODF TC for more than 4 years, yet no solution based on an approved standard (other than Open XML) has been found. These are all indisputable facts.
In his post, Rob proposes using "legacy OO namespaces" (also declaring OpenOffice as the "current convention"). Rob's suggestion to use "legacy OO namespaces" is a reference to a vendor's product and indicates favoritism to a particular implementation. The defender of "precise, repeatable, common" seems to be abandoning that hill, hoping instead to claim for his own the dialog that Microsoft has been conducting for a long time: Interoperability requires the participation of many, and will not be defined by a standard alone. Doug covers that pretty well I think.
The irony isn't lost at all. This is the same guy who went to such a length to chastise Open XML for its undefined list styles and compatibility settings. For some reason his expectations of Open XML seem to be somewhat higher than they are for the committee he chairs. For some reason, it is ok for Rob to patch glaring holes in ODF as "current convention" and then complain vigorously about alleged dependence on Microsoft Office for implementing Open XML. This is shameful, hypocritical and warrants corrective action.
It wouldn't be such a huge deal if the tone were constructive or aimed at improving the situation. It seems he is only interested in distancing himself from scenarios where ODF can be used successfully with Microsoft Office (as well as the DII discussions where that implementation was discussed in detail during its development. Funny that he didn't show up there to share this feedback.)
Rob's conclusion on the cause of that problem:
"I was taught to never assume malice where incompetence would be the simpler explanation. But the degree of incompetence needed to explain SP2's poor ODF support boggles the mind and leads me to further uncharitable thoughts. So I must stop here"
Let's just remember that it was the ODF TC which deemed formulas "out of scope," and after 4 years, still have no solution for standardizing the definition of "Sum = 2+2." Rob says "Everyone knows what =A1+A2 means." Really Rob? What does it mean if A1 contains 1, and A2 contains "two"? Would it surprise you to learn that Excel and OpenOffice produce different answers in that case? Which one is correct? This question and a thousand more like it is why formula interoperability is hard work, and not at all the trivial matter Rob claims it is.
During the original discussion within the ODF TC, not everyone agreed with the omission of formulas from the spec… David Wheeler seemed to be pretty clear when commented on this on February 7th, 2005:
This previous comment scares me: "There are from our point of view also no interoperability issues, because the namespace prefix mechanism we have specified unambiguously specifies what syntax and semantics are used for a formula". Here's how I read that: "Every implementation must reverse engineer all other implementations' namespaces (they're not in the spec, so everyone's free to invent their own private incompatible namespaces). Then, every implementation must implement all the syntax and semantics of all other implementations' namespaces for formulas, if they wish to achive interoperability. And oh, by the way, your implementation might not implement the namespace for the document you're trying to load, so you may lose all the formulas."
I'm sure that's not what was meant, but that's how it reads to me. I hope that helps explain why I think that the current formula information in the OpenOffice specification is grossly inadequate."
So… maybe it's too easy, but "I was taught to never assume malice where incompetence would be the simpler explanation." David Wheeler saw this coming over 4 years ago, and yet, OpenFormula is not a standard today, and ODF has no definition for spreadsheet formulas. Rob tries to excuse his way around this in his post, but these comments are made by the committee that he chairs. I'll leave it to you, then, to decide between "malice" or "incompetence" of the poster who would elect to throw his own committee under the bus to get hits on his blog… or fail to take this very good advice.
By the way, it is worth noting the response to this stern (and very accurate) prediction.
"Hi David,Thanks for the concerned comments and all the considerable effort you have put into solving this problem. You're challenging us all to go where none have dared tread before. So go ahead and lead the way. You have the TC's attention. We are listening. As you grind out the grit of your proposal, please keep in mind that we have to fit proposed solutions into the politic of work that has already been done. A politic that represents years of work that is just now on it's way to ratification at OASIS, and beyond to ISO. Keep in mind also that the ISO certification comes at the request of the European Union. Time is of the essence. Ratification perhaps trumps perfection. At least for the moment."
This comment was from Gary Edwards, (he of "cracks in the foundation" / OpenDocument Foundation fame) who eventually left the TC and shuttered the OpenDocument Foundation. I seem to remember some dialog from Rob about Open XML being "rushed" through standardization. Funny how those things come back to haunt you.
I'm very discouraged by Rob's post. As far as I can tell, rob is playing a shell game where only his definition will be good enough for supporting ODF, and that definition will change to whatever Microsoft isn't doing.
This is far from constructive. This is not a way to foster interoperability and industry dialog. This is not a leader for people to follow.
mad hatter asked:
"Gray, I would like to ask why Microsoft was the only organization who's documents were so badly out of compliance in Rob's tests? You said it's because the ISO standard isn't detailed enough, but if that was so, we would expect Open Office and KOffice documents to not be compatible, and they are."
gray knowlton answered
"@Mad Hatter... if you read the posts, you'll see the problem. Because ODF does not define a syntax for spreadsheet formulas, implementers are left to choose how to represent them when writing the format. I suggest you also read http://blogs.msdn.com/dmahugh for a little more detail. "
This is not an answer. He is asking why are you the *only* different and no interoperable implementation. Do you thought in ODF users when you took this decision or was a pure strategical/marketing drived one?
Marc, the ODF 1.1 implementation in SP2 is the *only* ODF 1.1 implementation that uses a namespace prefix (as specified in the spec) to define the syntax and semantics of its formula markup through reference to an approved, published standard.
OpenOffice and KOffice use an undocumented non-standardized extension to achieve interoperability, which will no longer be allowed in ODF 1.2. We've heard loud and clear for a long time that the ODF community does not want Microsoft to extend the standard, and we've followed that advice in SP2's implementation.
I tried the formula-thing in Gnumeric creating and saving a simple spreadsheet. The result: Gnumeric saves formulas in the interoperable way ( contrary to what the Microsoft product does ):
~ $ gnumeric --version
gnumeric version '1.8.2'
Doug, in ODF 1.2 conformant documents, the used formula *prefix* will have to be "of:", and none of the current OOo or SP2 prefix will remain valid.
OOo and Koffice DON'T use a non-standardised *extension*: they simply use an allowed prefix, exactly in the same way as SP2 !
What you explain above as being the Microsoft excuse is simply a lie !
Doug, after reading again your comment, I now see what you meant by "undocumented non-standardized extension to achieve interoperability". Please replace in my comment above the word "lie" by "far fetched explanation". Apologises for an inappropriate word.
I was confused by the fact that, while standards are normally made to improve interoperability, Microsoft manages to use standards as an excuse to break interoperability !
Doug, one thing puzzles me. You wrote that SP2 "uses a namespace prefix (as specified in the spec) to define the syntax and semantics of its formula markup through reference to an approved, published standard."
Isn't it strange that the prefix used by Microsoft to reference this standard is "msoxl"? I would have rather expected that you use something like "ooxml". This would have sound more ISO-29500 than the proprietary prefix "msoxl" like in "MicroSoft Office XmL"...
"Isn't it strange that the prefix used by Microsoft to reference this standard is "msoxl"? I would have rather expected that you use something like "ooxml". This would have sound more ISO-29500 than the proprietary prefix "msoxl" like in "MicroSoft Office XmL"..."
I never actually thought about that. One thing that might speak against using "ooxml" or "is29500" or similar would be, that the namespace used for spreadsheet formula in SP2 is not defined in OOXML, so it might cause a bit of confusion.
So rob weir actually refused a beta version of the specification ?
Could he explain what possible real reason he could have for doing that.
The license issue seems moot as millions of people use MS beta products every day without any problem and have done so for ages. and that included a ton of IBM people as well as I have seen very often as well. (I actually recieved an Office SP2 beta copy from an IBM guy).
As ODF TC chairman getting the chance to preview the ODF implementation of the leading Office product suite should be quite normal. And even in a role of IBM office product representative involved in producing new IBM Office prodcuts he and/or his IBM collegues should have been evaluating MS Office SP2 for at least 6 months or so.
There is just no way in hell that Rob Weir only knew of interoperability issues untill the official release of MS Office SP2.
Most likely his comments were timed to a moment that the Symphony 1.3 beta was able to show compatibility with OpenOffice.
It is a total disgrace that Rob Weir apperantly refuses to test a beta product of the leading Office suite he was offered to do so and then critisises this product in a post where he actually compared it with non public IBM 's beta products that he never prior to his post offered to scrutiny or commenting by others.
Can anybody seriously persieve IBM development team fully ignoring devlopment of the leading spreadsheet product in producing their own new product and not testing interoperability themselves using the MS Office SP2 beta ?
If so I cannot see IBM as a serious Office developer but in reality we could all easily see that IBM (and Rob) were already aware of the interoperability with MS Office SP2 for months if not even already last year.
The currently available Symphony 1.2 versions actually mutilates OOo 3.x files in a way that is horrible to any users making the information in cetain formulas unavailable and this makes interoperability a total joke.
Compared to IBM's current Symphony implementation the Microsoft approach which micht not preserve formula's but only values represent a much more valued approach as the reciever will at least have a spreadsheet with reliable data preserved for the user, something that IBM's product is unable to accomplish.
[quote]And exactly this is already done for example with CSS and (X)HTML. So why not have a similar validator for ODF? Maintained by OASIS, just as the W3C is doing?[/quote]
Actually the MS Offfice SP2 files seem to validate fine against the OASIS ODF schema's.
A validator would not be of much use on ODF on such issues as it has several crucial parts not validating. Both formula's and MathML 2.0 is not in the schema's.
Formula's in ODF are just arbitrary strings in the XML schema's and MathML even is arbitrary markup in the ODF schema's.
The latter being helpfull to OOo as including MathML 2.0 schema's or use the official schema's as a normative reference would make all OpenOffice files using math fail.
"If so I cannot see IBM as a serious Office developer but in reality we could all easily see that IBM (and Rob) were already aware of the interoperability with MS Office SP2 for months if not even already last year."
Actually, I am sure they have been aware of how Microsoft Office 2007 SP2 handles formulas in ODF since at least August 2008. I documented the approach by SP2 after the first DII-workshop when I wrote the article "DII ODF workshop - the good stuff". This was August 18th 2008. At that time we were playing with a pre-alpha-release of Microsoft Office 2007 SP2 and two of my conclusions were these:
1. Microsoft Office 2007 SP2 uses their own formulas in ODF spreadsheets
2. Microsoft Office 2007 SP2 strips away formulas from "unknown" namespaces
(the last test file in the article, Testfile_20.ods)
@Jesper: "One thing that might speak against using "ooxml" or "is29500" or similar would be, that the namespace used for spreadsheet formula in SP2 is not defined in OOXML, so it might cause a bit of confusion."
There is clearly a serious contradiction here: Microsoft insists that they use the "only standardised formula syntax", but then it appears that the namespace is not defined in the standard, and Microsoft has to use a proprietary namespace to implement the "standard" syntax.
The impact is that if somebody else (e.g. OpenOffice.org) wants to be interoperable with the ISO OOXML formula syntax, it has to use the Microsoft proprietary namespace ! This is a nonsense.
Isn't it an additional proof that OOXML is much more a proprietary Microsoft standard than an open ISO standard ?
At least, it is a proof that more time was needed to correctly review OOXML before making it an ISO standard. (Another proof of this is that there is no mechanism in OOXML to identify the version, but this is another story)
"There is clearly a serious contradiction here: Microsoft insists that they use the "only standardised formula syntax", but then it appears that the namespace is not defined in the standard, and Microsoft has to use a proprietary namespace to implement the "standard" syntax."
Excuse me - but you seem to be completely missing crucial technical points here.
Microsoft uses the formula syntax in OOXML - the only standardised formula syntax. The format is perfectly (well, ahem) defined in ISO/IEC 29500. The formulas are part of the specification of SpreadsheetML in Part 1. They are defined in BNF-notation in section 18.7 (Formulas) from page 2268.
About the namespace: it's a pointer to where information about how to interpret it can be found. As Doug wrote somewhere, ODF requires the following for formulas:
"a namespace prefix specifying the syntax and semantics used within the formula."
And this is exactly that Microsoft does - they supply a namespace prefix (msoxl) while explaining in their implementer's notes that this refers to Excel's formula syntax. Excel's formula syntax is defined in ... tadaa ... ISO/IEC 29500.
(I do think that they should elaborate a bit more in their notes - explaining exactly where the syntax and specification of Excel's formulas can be found - I'll log a comment on this in a bit)
"Isn't it an additional proof that OOXML is much more a proprietary Microsoft standard than an open ISO standard ?"
No - it's additional proof that you don't always know what you are talking about.