Thinking the unthinkable, pondering the imponderable, effing the ineffable and scruting the inscrutable

Cracks in the Foundation

byRobon2007/10/07

You must admire their tenacity. Gary Edwards, Sam Hiser, and Paul E. Merrell (aka “Marbux”) . The mythology of Silicon Valley is filled with stories of three guys and a garage founding great enterprises. And here we have three guys, and through blogs, interviews, and constant attendance at conferences, they have become some of the most-heard voices on ODF. Maybe it is partly due to the power of the name? The “OpenDocument Foundation” sounds so official. Although it has no official role in the ODF standard, this name opens doors. The ODF Alliance , the ODF Fellowship, the OASIS ODF TC, ODF Adoption TC (and many other groups without “ODF” in their name) have done far more to promote and improve ODF, yet the OpenDocument Foundation, Inc. seems to score the panel invites. Not bad for three guys without a garage.

However, in recent months the OpenDocument Foundation has found itself more and more isolated, outside of the mainstream debate. How far they have fallen can be seen in the fact that Microsoft has gone from ridiculing their conspiracy theories to using them to support their arguments. At the same time the Foundation’s membership has dwindled to the point where only a small number remain. Former members have disassociated themselves from the Foundation as it turned increasingly to strident rhetoric. Whereas in the early days, the Foundation had a large membership that participated fully in the OASIS TC’s, now their “contributions” are mainly that of heckling and haranguing the other members. Finally, the Foundation has recently announced its intent to abandon constructive work within OASIS, to actively lobby against adoption of ODF 1.2 in ISO and to push for an alternative format, CDF, based on XHTML, CSS 3.0 and RDF. This is an odd stance for a non-profit whose charter was:

The OpenDocument Foundation, Inc. is a 501c(3) non profit chartered to work in the public interest to support, promote and develop the OASIS OpenDocument File Format affectionately known as “ODf”.

So it is against this backdrop that I read with interest in Linux Today the latest correspondence from the Foundation. You can read it yourself, or take the following 8 points from me as a condensed summary of their main points:

“The commercialization of interoperability remains a key driver in both big vendor deals and big vendor consortia FOSS is left on the outside looking in.”

The conversion to XML [document formats] must be nondisruptive” meaning it fits into existing business processes which are increasingly dominated by Microsoft middleware. This implies a requirement for high-fidelity, loss-less round-trip conversions.

The alternative is “rip and replace” and that is too costly and disruptive.

Microsoft is moving toward a “grand convergence” of their services, desktop, device and servers, with OOXML at the core. “MS-OOXML is the primary transport, the document/data container of interop-integration preference.”

ODF was not designed as a response to these problems.

Microsoft/Sun/Novell are working “to limit ODF interoperability and usefulness” because of some patent deals. (Sorry I can’t summarize this one better — I just don’t understand it.)

IBM/Oracle/Google are working to “limit ODF interop” because “they want a total ripout and replace of MS Office.”

The Open Document Foundation is in “the middle area of trying to perfect the conversion to XML”.

Let me take these points one-by-one:

The OpenDocument Foundation seems to try to clothe themselves in the mantle of the open source community and pontificate on how the big bad vendors treat interoperability. But are they speaking as a non-profit or as a vendor? Take their DaVinci plugin, for example. Where is the source code? Why isn’t this open source? Are we to follow the Foundation’s claim of 100% interoperability, based on blind faith, without seeing some proof in the form of working code? I’ve been working on document conversions and document file formats of one kind or another for almost 20 years. I’ve never seen 100% fidelity conversions of anything but trivial formats. Extraordinary claims require extraordinary evidence. But we have nothing here, just white papers.

I would not claim a priori that all customers require lossless, 100% fidelity conversions. Remember, we do not see 100% fidelity even when upgrading from Office 2003 to Office 2007, but this appears to be adequate. What is required is that the total return from changing document formats exceeds any other profitable use of capital available to the enterprise. In other words, to a business this is an investment, and will be judged as an investment. Very few businesses will take a dogmatic, ideologically pure view of this. Ask yourself, would you accept 1% loss in fidelity if I gave you a billion dollars? Yes,of course you would. There are no purists in business who will remain in business. We’re just haggling over what price/fidelity combination is needed to make a prudent investment.However, there is a notable exception to this rule, and that is where access to open document formats are mandated as a public right, not as a business investment. Think of the last 20 years or so of enabling public buildings with ramps for the disabled, bathrooms to accommodate wheelchairs, braille lettering in elevators. This was done by legislation and regulation, as a matter of public policy, to ensure that all of the public has access to public facilities. There was no requirement that an access ramp post a net profit. Similarly, today we see some movements to ODF are based on open-access principles.

This is what we call the “fallacy of the excluded middle.” You are either with us, or against us, etc. It is false to suggest that the only two approaches to interoperability are to either blindly follow the OpenDocument Foundation’s mysterious DaVinci plugin, or to ignore interoperability altogether and advocate rip and replace. There are today two other other ODF plugins available, one from Microsoft and one from Sun. This is real, running code, open source even in the case of the first plugin. So why should we be taking exclusive direction from the Foundation on how we achieve interoperability? Oh right, they are claiming that their program achieves 100% round-trip fidelity. Extraordinary claims…

Gary is in the ballpark when he suspects that Microsoft is seeking some sort of “grand convergence” around protocols and formats. However, I disagree with his impression that OOXML sits at the center of this. In my opinion, OOXML is a rushed, transitional format, intended purely to disrupt ODF adoption. Just as the Office 2000, Office XP, and Office 2003 markup formats were abandoned by Microsoft, I predict that OOXML will soon be cast aside. The problem is that OOXML is such a poorly-engineered format that not even Microsoft wants to build upon this. If I had to divine the future of Microsoft’s file formats, I’d look more in the XAML/XPS/Silverlight space. I believe that future MS Office document formats will look more like that than like OOXML.

I find this observation amusing. ODF, which started its standards track late in 2002, was not designed to be 100% compatible with Office 2007. Mercy me, how did we manage to drop the ball on this one?! Remember, in 2002 there was no publicly available specification for Microsoft document formats. There was no Open Specification Promise or Covenant Not to Sue. So not only was 100% compatibility technically impossible, attempting it via reverse engineering was precarious from a legal standpoint. In my opinion, it still is, even in 2007.In any case I’m staunchly opposed to evolving any open standard purely for the benefit of a single vendor. Microsoft Internet Explorer is the dominate web browser. Should we then require that HTML only evolve in ways that improve interoperability with Internet Explorer? I don’t think so. Why should document formats be different?

This comment manages to avoid confronting a heap of contrary facts. Microsoft supports the open source ODF Translator project on SourceForge. Sun has made their own ODF Plugin 1.1 for MS Office available for download. And Novell, along with helping the Microsoft effort, has integrated that translator into their version of OpenOffice and has also started work on more powerful, next-generation support for OOXML. So these three companies are seeking to “limit ODF interoperability and usefulness”? If so, they sure have a clever way of disguising their intent. To the ordinary bystander, writing conversion and translation code to allow documents to be shared between OpenOffice and MS Office would be seen as a pro-interoperability statement. But thanks to the OpenDocument Foundation’s in-depth sleuthing, we now know that the opposite is true. Not!Although I have serious doubts as to long-term technical feasibility of some of these translation endeavors, they do have the advantage of showing real, running code working with real, running applications. They may not claim 100% fidelity, but this is first-generation work and will undoubtedly improve. But they have an important advantage over the Foundation’s DaVinci Plugin in that these other efforts demonstrably exist. Given a choice, I’ll take an open source version of a partial fidelity convertor, with a reasonable architecture, over one that claims 100% fidelity, but that I can’t see or touch.

The claim is that IBM/Google/Oracle also want to “limit ODF interop” because (according to Gary) we want rip & replace. Strange, but just a few weeks ago I lead an ODF Interoperability Camp in Barcelona, on behalf of the OASIS ODF Adoption TC, where we had a good selection of ODF vendors, open source projects and customers working to improve interoperability, including Sun, Novell, Google and IBM. The OpenDocument Foundation is a member of the OASIS ODF Adoption TC. So did they help in the organizing of the event? Did they participate? No, nothing, nada. Evidently it is easier to complain about interoperability than to do something about it.And again there is this fallacy of the excluded middle. You must either accept the magical DaVinci Plugin, or you are for rip & replace. There are no other alternatives considered. I’d remind the OpenDocument Foundation that interoperability was not invented yesterday, and that there are many technical approaches that can be applied to foster it. Open standards are one way, but there are others that can be applied as well, including conformance testing, test suites, plug-fests, profiles, shared code, reference implementations, etc. We should apply experience and engineering judgment to select the appropriate solution for the problem, and not fall into the trap of believing that there is only a single path to interoperability, and that this path just happens to be based on the Foundation’s product.

Although it sure would be nice to portray yourself as the little guy, watching out for the customer, while the big bad vendors tromp all over the flowers, the fact is that the big vendors are actively working on interoperability, with at least three major solutions available today, as well a major initiative around interoperability in the ODF Adoption TC. In particular, IBM (with SmartSuite) and Sun (with StarOffice) have 15 or so years experience each in working on document interoperability with MS Office. This isn’t rocket science, but neither is it easy. You can either stand on the sidelines and make pronouncements about how the world is out to prevent interoperability, or you can roll up your sleeves and help get the work done. I know which one I’ll be doing. What about you?

If the Foundation’s approach was technically feasible, they would just go out and do it. You don’t let a breakthrough technical innovation wait on a standards committee to act. You just go out and do it and then standardize it later, once you’ve proven it works. If the Foundation really thinks that they can achieve 100% interoperability with MS Office with just 5 simple changes to ODF, then why the heck don’t they just do it? Don’t wait for the formality of an the ODF TC ‘s approval. They should go ahead, as if the standard already had their 5 fixes, and show the world how they have achieved 100% interoperability with MS Office. If they are right, they would all become multi-millionaires in a very short period of time.

Thanks for posting in a long, specific way that people will actually read and listen to exactly what I’ve been saying since I read the last few PR statements from the OpenDocument Foundation. Now I can link here when I’m trying to make my points.

Nice job of spinning our position into something you can deal with, Rob. But to cut to the chase, ISO/IEC JTC 1 Directives require that: “Standards designed to facilitate interoperability need to specify clearly and unambiguously the conformity requirements that are essential to achieve the interoperability.” http://www.jtc1sc34.org/repository/0856rev.pdf pg. 145.

Got a link to those interoperability conformance requirements for the OpenDocument specification, Rob? Why not? Might it be because they don’t exist?

It’s good to see IBM finally talking about OpenDocument interoperability issues in public. But I invite you to focus on the issue of the missing interoperability conformance requirements .

I always believed that The OpenDocument Foundation was a non-profit, open source, organisation with an official role in promoting the OpenDocument format. This is certainly how The OpenDocument Foundation has been promoting itself.

If the OpenDocument Foundation has no official role, then the owners of the OpenDocument trademark should not permit its trademark to be misused in this way any longer, because it is misleading and damaging to the OpenDocument format. The name “The OpenDocument Foundation” is absolutely misleading.

I can’t help wondering if there are ulterior motives to The OpenDocument Foundation’s recent moves. First, have they been bought by Microsoft? If I was Microsoft, and I looked at The OpenDocument Foundation, and I saw that it was an oficial looking but proprietary company that could be bought in order to attack ODF, then I would be 100% certain to buy into it for this purpose.

I also wonder if ODF 1.2 with it’s foreign element tags would allow others to bypass it’s proprietary DaVinci plug-in for MS Office with better implementations, and whether this is the reason for their sudden switch.

Unlike most of my blog postings, I took no pleasure in writing this one. In fact it was quite painful for me to write. I know that you and Gary have, in the past, made many, many contributions to benefit ODF.

However, you are increasingly boxing yourselves into a militant, anti-ODF position and spreading all sorts of zany conspiracy theories, and it is even more painful to listen to and read those. So I’m no longer going to let my respect for your past contributions blind me to your recent errors. Of course, it is a free country and you can say whatever you want. And so can I.

As for conformance clauses, this is a fair point. OASIS recognizes the need for formal conformance clauses, and as of this summer all OASIS standards, even those not going to ISO, will require a formal conformance clause. So this is something we will need to write up for ODF 1.2. The contents of such a clause is a good thing to be debating in the TC, or even publicly.

I was reading this in the unread-items folder of my feed reader and I had no idea who was speaking. I didn’t figure it out. I had to see your signature at the end. Lovely, just lovely. As a bystander, you make more sense of this than anything else I’ve seen.

One nit: I don’t have the same sense as you about XPS (an OPC-carried final form that may have more legs in high-end on-demand publishing than PS and PDF do) and the WPF thrust of XAML, Silverlight, and such. It will be interesting to see.

When I first saw the daVinci bluster, along with some rather cynical whining, I sealed a prediction in an envelope and mailed it to a friend. I’m not ready to have it opened, but I bet your prediction would not be much different than mine.

I’m also pleased to see you mention ODF 1.2. I don’t see much on the site, and I wonder what is going on with OpenFormula. There has not been a new draft in some time. Is it soup yet and what ODF version is it expected to appear in?

I’m from Groklaw; I respect Marbux a great deal. However, if he’s joined forces with Gary Edwards – apparently, I don’t really know him. Gary and his 100% fidelity plugin that nobody has ever seen have appeared to me to be rogues in this process.

Note that I am not opposed to having a 100% fidelity conversion plugin. My beef is that if you have one, you should be distributing it. It should be possible to be seen by people. From what I’ve been able to find out about Gary Edward’s converter, I have to assume it’s a 0% fidelity converter, because I have seen a grand total of zero conversions it produced.

Marbux, please reconsider with whom you work, because if you work for someone who appears to be deceptive, you get a reputation for deception, even if you do not do any yourself. In my opinion, your reputation is clearly being damaged by this relationship.

IIRC, the plugin does exist and was demonstrated at one conference, much to the astonishment and delight of all in attendance.Since then, nothing. I don’t see anything to download except a proof of concept at what I presume to be the foundation’s home page.

I’d love to see a beta of the plugin, and some reassurance that it will be open source.

Yeah, I was also once affiliated with the Foundation, and observed firsthand the events that Gary and Marbux continue to complain about. I don’t really accept even the factual basis of much what they argue, much less the conspiratorial interpretation.

As Rob says, the vote on the list proposal reflected a consensus opinion of the TC, and the proposal that won was in fact most directly developed by and vigorously promoted not by Sun, but rather the KOffice developers; you know, exactly the kind of free software effort Gary and Marbux promote.

As for the problems with OASIS, I really don’t see the conspiracy. I fault OASIS for being a little dim on the politics of the whole rules issue, and Gary for being careless with those rules, and with not informing those of us most affected by them. In the end, a number of TC members (from IBM, Sun, Adobe) stepped up to get those of us actively contributing free memberships.

It’s also worth noting that two of the biggest new additions to ODF 1.2 (metadata and formulas) were primarily driven by independent developers unaffiliated with the big vendors.

As for what Rob said about the hypocrisy of trumpeting open source and interoperability but not releasing the source for the fabled plug-in, what he said. I’m confident we’ll never see it, but would be happy to be proved wrong.

Microsoft says this today – before the standard is even approved and it’s important for Microsoft to look as standards-lover. Do you have nay doubt that when OOXML will be approved they’ll change the stance to even more aggressive one ?

So, the Foundation are no longer interested in OpenDocument. I hope OASIS enforce their trademark and stop them using it. They can be the Compound document foundation if that’s what they want.

Happily they’re doing to save us all from interoperability disaster with their buzzword-friendly CDF+ format with no “eXtensions”. What’s the betting that – just like “ACME 376″ or whatever it was – this is just RTF transliterated into XML and a little bit of spice.

CDF is just a container, like a Zip file. They are notably quiet on what the CDF file _actually contains_.

The government are very keen on amassing statistics. They collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But you must never forget that every one of these figures comes in the first instance from the village watchman, who just puts down what he damn pleases.— Sir Josiah Charles Stamp