the Complexion Connexion

************************UPDATE: Florian's take on the anti-Foundation PR campaign is humorous and, typical of his incisive wit, guns down the contradictions in the ODF community's position.************************

The message of the day -- give or take -- reads: 'OpenDocument Foundation is helping Microsoft and Microsoft OOXML by promoting the CDF format in lieu of ODF!' 'Great Rift Seen!' Whatever.

This is unfortunate for the ODF community because we, and CDF, relish a stage we were not intending to take. In short, if the ODF community is looking for who's helping Microsoft, they needn't look far.

You hardly need a link since most of y'all are already on these feeds...

********************************Mary-
As we've pointed out to our colleagues & friends in the ODF community, both OOXML and
ODF v1.2 are having (going to have) difficulty at ISO.
ODF v1.2's problem is that it doesn't respect ISO's "JTC 1" directive
mandating interoperability. Same OOXML.

So, the dream of a single universal document format (around which all
applications have equal access) has, for the time-being, gone sideways.
We tried really hard working within the standards process (for years)
and our interoperability proposals were rejected by Sun (who leads the
ODF OASIS TC by governing the schedule of what proposals will be
discussed and what and when voting will or won't occur).

They read our
whitepaper published in Spanish in Dec 2006 ... "Interoperability: Will the Real Universal File Format Please Stand Up?"
... where we first clearly articulated our vision for ODF v1.2 and the
necessary changes to the ODF specification to align it with ISO interop
directives as well as the high-level of document fidelity required by
the enterprise market (simply installing OpenOffice is not enough).

Experts were able to grok that the OpenDocument Foundation's interop
proposals were necessary also to make the Foundation's ODF Plug-in
viable at the ODF format level. We knew it would take Sun's Hamburg
engineers 2 to 3 years to catch the OpenOffice application up to our
proposed changes in the ODF spec, but hey, that's life. Sun blocked the
changes we believe to avoid embarrassment if the ODF spec should be
able to do things that the application could not handle; and they did
it also to honor their $2 Billion hardware & server commitment to
Microsoft (in the 2004 pact, meaning to prevent or delay perfect
interop with MS documents) and, therefore, to keep the Foundation's
plug-in from the playing field.

All this is merely historical arcana to many observers, and the open
source chorus seems to think it's a good idea to avoid extending the
life of Microsoft's legacy applications with an EXCELLENT-QUALITY
(**INTERNAL**) plug-in ... only weak plug-ins (Microsoft's, Novell's,
Sun's) are wanted.

What seems missing from the discussion is that the market is going to
give the de facto
standard back to Microsoft, regardless of events at ISO. That is,
unless we provide a clean way for enterprises to access their Microsoft
formatted documents and play with an open document format standard
within BUSINESS PROCESSES.

While part of the world is swallowing the ODF story, those enterprises
who are testing ODF implementations and hitting the brick wall of
business processes (Mass ITD [with whom I am under NDA], Denmark,
Belgium) are finding it impossible to implement across autonomous
networks of decentralized government agencies (each with its own CIO)
for practical reasons. People wonder why the ODF policies are drifting
back to ODF + OOXML. Wonder no more.

It's true -- I was a vociferous supporter of ODF, and among the most passionate and committed people to identify the importance of OpenOffice's file format back in 2002.
But this year the vendors' tendency to use open standards and open
source applications as a bargaining chip to extract (some would say
'extort') money from the deep-pockets Gorilla [that's your client,
Microsoft] seems to have won out over good software that works for
people.

CDF is better. It's governance is with the W3C: does anyone dare
question the integrity of Sir Tim? And it is a format -- or I should
say a framework for open formats -- which does something ODF can't do
at all and OOXML can only do with Microsoft products -- it can go
Mobile.********************************

For one thing, it would be impossible to help Microsoft get the 1,000-odd unique comments through a five-day Ballot Resolution Meeting process in February at ISO. I'm not sure what force of nature could possibly help that, unless convener Alex Brown is seen along the quaint cobblestoned environs of the Bristol quays anytime soon driving an Aston Martin DB9. Microsoft will undoubtedly fail to get ISO approval for OOXML. They don't need my help, either way.

(And if by some sleight of hand they do get OOXML approved, ISO will go into such a spasmodic reform mode that there will never be another important standard from Microsoft.)

Among our difficulties with ODF is that ODF v1.2 is headed for failure at ISO, too. And if the ODF community is tone-deaf to the necessary enhancements (which the Foundation has proposed), then we can't help ODF either -- much as we would like to and much as we have dedicated years of our lives in order to influence.

More to the point, it is our view that if you want to help Microsoft, you will do nothing to fix the interoperability problems of ODF; you will do nothing to help ODF address the enterprise market's requirements to work cleanly with the existing documents and business processes of almost a half-billion document authors; you will stop, obfuscate and delay the efforts or others to introduce the material to enable the ODF standard to interoperate with the installed base of MS Office documents. This is what one does to help Microsoft.

Back in the languid torpor of summer, Sun's John Bosak made the statement explaining Sun's puzzling vote at the American standards body, INCITS, a vote which benefitted OOXML's case in the run to the failing round in September ("Black September"). His confusing announcement can be read in its moment as a message to all NB's that they should follow Sun's lead to submit "Yes with Comments." Bosak made this declaration in full knowledge that only "No with Comments" can force ECMA to address the comments in the later ballot resolution phase. Bosak knew that a "Yes with Comments" was a piss in the wind.

We wish to make it completely clear that we support DIS 29500 [OOXML] becoming an ISO Standard and are in complete agreement with its stated purposes of enabling interoperability among different implementations and providing interoperable access to the legacy of Microsoft Office documents.

At the recent GOSCON panel on ODF, Sun's Doug Johnson joined Microsoft's oxymoronic argument in favor of multiple formats, including OOXML, because of the market requirement to interoperate perfectly with legacy Microsoft documents. (This left IBM's representative, Arnaud Le Hors, stunned as he was surprised to be caught between Sun's and Microsoft's bullshit sandwich.)

If you parse their actions in the development of ODF at OASIS, you would actually understand Sun's position is that full high-fidelity interoperability is "outside the scope" of the ODF specification. (I have links for this on the OASIS mailing list archive if you'd like.) IBM for reasons I grope hopelessly to understand has acquiseced in this deception.

That's helping Microsoft. With help like that, there's not much more rope left for us poor sods at the Foundation to pull on.

It is beyond question that no organization -- except perhaps SCO, Novell or Microsoft themselves -- has done more to help Microsoft than Sun Microsystems in their conduct of the leadership of the OASIS ODF TC and their failure to encourage the kinds of innovations at the file format specification level that would make ODF a legitimate universal document format. I'll leave you to speculate what connection the Sun-Microsoft pact of 2004 may be having here.

If the Foundation were on record, we are more into Microsoft being drummed out of town -- or beaten 500 million to nil in a boisterous market referrendum. Meanwhile -- while we wait for the Obama Administration to create the necessary political will to drop the RICO statutes on Microsoft -- we'll just have to go and Embrace & Extend the Microsoft installed base by going and getting those files and piping them into trustworthy open standards (standards with process & governance integrity) with unimpeachibly open business processes surrounding them.

We are a single-format shop. Our interests have shifted to the W3C's Compound Document Format. Our ideology, if you must, is for a universal document format about which any and all applications can work with equal rights. Jason Matusow says our actions tell that we are for multiple formats. Jason's full of shit.

We will of course support ODF whole-heartedly if the specification improves in the necessary ways, ways about which we have been clear and will be happy to be clearer in future if we have lacked a certain, shall we say, puissance of message. I am not sanguine, though: I don't expect ODF or ODF's governance environment to improve in the time-span necessary to stop Microsoft from cementing Exchange|Sharepoint and Sharepoint|Exchange Live Whatever across most of what's left of the Lotus Notes installed base.

No, I'm not encouraged that IBM has taken over OpenOffice.org's development. This merely means that Sun has just sold IBM a condo in Corral Gables with a 180 view of the parking lot of the Kingdom Hall Bowling Lanes, Barbecue & Pet Salon and Sun Microsystems is moving on to bigger business (business that actually has money) as a thinly disguised Microsoft subsidiary selling Windows-primed servers and killing Linux with "Solar-Ian" (Ian Murdock's Super Duper Linux Application Layer Embedded in openSolaris) Which-Interoperates-By-Design-With-Microsoft.

Accordingly, IBM better get busy acquiring Novell before Novell becomes a thinly disguised Microsoft subsidiary dedicated to implanting .Net and patent-encumbered dependencies -- or the idea of patent-encumbered dependencies -- in Free Software -- I say this much as IBM detests the GPL.

If helping half-a-billion souls get away from Microsoft is helping Microsoft, then I'll be darned.

Denmark's was a TWO STANDARDS position (paradoxical as that may be) for evaluating both OOXML and ODF; yet now in the echoes of Swedish standards process tampering by Microsoft, the Danish ministers are asking for a technical assessment of OOXML to be advised by the government technology ministry so that Standards Denmark may make its September 2nd (Monday) judgment on OOXML based upon OOXML's actual technical interoperability.

This is odd this late in the game. It suggests to me that the Danish government was taking Microsoft's word on OOXML's interop qualities at face value. Conversely that means the Danish government was taking our claims about OOXML's lack of fitness as rhetorical IBM-based corporate gamesmanship.

Now, the events in Sweden (Microsoft offering payment to partners to attend meetings and vote) puts OOXML's quality into question for those who have not read the technical specification yet or trusted the transparency of earlier analysis. Sweden illuminates that Microsoft has no faith in the technical merit of OOXML and feels the need to purchase its passage -- even with every other advantage attending its ISO Fast-Track.

I'd like to hear John Gotze's view from Denmark...unless that's what's up on Groklaw ;-)

Big news in the merger space this week: Denmark -- the one just north of Germany -- has given over its government to an American corporation.

Countries have been co-opted more or less whole by corporations before -- notably the USA by Haliburton, Chinese weapons manufacturers in Sudan, de Beers & South Africa, The East India Company and the whole world east of Mauritius -- but what's new is that Denmark, what we had assumed was an advanced Northern European country with a more or less open & democratic mode of government, is paying Microsoft in what appears to be a reverse-acquisition for the privilege of having that corporation take over Denmark's national policies.

"It's in fast track at the moment | and that's a process | you know we must be | we have to now | we've given | Microsoft has given | the business leaders in the company understand -- okay? -- that the control of the file format has been given from Microsoft to Ecma International."

This will be the first policy out-sourcing and the brilliant new technique looks likely to cascade throughout the European Union. Despite attempts by the EU to fine Microsoft hundreds of millions of Euros for not complying with its requirements of honest corporate behavior, it looks as though Microsoft will be able to fund indefinitely those fines through sales of its new Office software because Europeans like to buy new shiny things (with difficult packaging) from America -- even if they do not offer anything actually new.

Joe Wilcox over at Microsoft Watch is the first media analyst to offer clear insight -- "When is 'Open' More Open for Microsoft?" -- as to why the MS OOXML file format is driving Microsoft so insane to get approval from the International Standards Organization (ISO)...

OOXML has already received Ecma
standards certification, but needs ISO approval, too. Approval would
bring OOXML on a standards level with competing ODF and could help
advance the Microsoft format's customer adoption.

The latter objective is similar to VC-1 and Silverlight: OOXML
would be more open for Microsoft than other developers. ISO
certification is important to Microsoft because:

OOXML is the default for Office 2007

OOXML is important for the advancement of other Microsoft strategies, such as business intelligence.

The format will plug into some of Microsoft's future work
around enterprise informational and Web services, collaboration and
content archival

OOXML is much more than a file format; it's being groomed for a platform role, similar to VC-1 and Silverlight.

This will explain in part why the 6,000 pages of the OOXML specification is such a strange read. Good get, Joe!

OOXML is really an insertion strategy -- a Trojan Horse -- for a proprietary XML platform on which much of Microsoft's next generation product catalog will depend.

Denmark? NOW do you want to sanction both ODF and OOXML for government business processes? If so, you will be falling into a trap. When a government approves a document format, it wants to approve a document format, not a Swiss Army Knife.

John Gøtze kindly brings to our attention a new development in the Danish government's progress toward open software standards.

Now, having gotten past the preliminaries of whats, wherefores, hows & whys, the Danes have progressed to the question of implementations. This always reflects a watershed moment for government leaders, staffs & citizens who will be effected. It reflects a body of people coming through a process of understanding and it shows their confidence.

This reminds me of nothing more than Peter Quinn's meeting of vendors in the Massachusetts State House in June of 2005 where he said,

"Open document formats: I get it! But how do I get there? Discuss."

The difference now is that we are dealing with a whole country, Denmark, which according to scale is something like the size of Massachetts within the larger body, the European Union. In contrast to the Massachusetts situation, other parts of the EU are already migrating ahead of whatever policies or regulatory guidelines are being established. These include agencies in France, the UK, Germany and Belgium among others.

There is an interesting similarity to Massachusetts. The policy memorandum, ETRM 3.5, which fostered the ODF debate there was similar to this Danish plan in its underlying motivations and intent.

When a government gets past identifying the ideal scenarios that are possible, those which exist and are ready to implement -- in this case they include open standards like XHTML, CSS, ODF -- and moves on to the questions of how to get them used, there is always a large number of impediments to the final result. These include recalcitrant software monopolies (who are constantly trying to undo good policy work), general inertia against change, decentralized structure of multiple agencies with different ICT systems, leadership and beliefs about what works, and the difficulty in establishing an authoritative, credible but also flexible recipe for pushing change without increasing cost, stress & disruption.

That's why these policy frameworks look so alike: every bureaucracy gets to this same difficult place eventually -- 'How do we get there?'.

Says John Gøtze...

The implementation plan is presented in a report which suggests that
“open standards should be implemented gradually by making it mandatory
for the public sector to use a number of open standards when this
becomes technically feasible”.

What has happened?

The existing Danish Interoperability Framework (in Danish) has become mandatory. Separately, the report lists a number of open standards which should be implemented by Jan 1, 2008, through the normal course of system upgrading (unless the transition is deemed disruptive). Gøtze mentions a few...

Standards for data interchange between public authorities

Standards for electronic file and document handling

Standards for exchanging documents between public authorities (Open Document Format and Office OpenXML)

We're interested here in ODF. Here's what the report says about document formats...

With regard to standards for exchanging documents between public
authorities, the report proposes that “it should be mandatory to use at
least one of the document standards Open Document Format or Office
OpenXML”, and that it is up to the individual agency to decide what
they want. The report explains that a study will be conducted this year
with “the purpose of obtaining the necessary experience with these
standards before 1 January 2008″.

So, the Danes are looking at a mandatory shift to either or both of the two XML-based document formats. You say ODF AND MSOOXML BOTH! EEEEEEK!

This gives me no anxiety whatsoever. MSOOXML has already been thoroughly de-bunked vis a vis its repudiation of other existing standards; it is in perhaps a perpetual deep-freeze at ISO (from which Microsoft will not seek or wish to remove it, since "ISO status pending" is all they need to sell it; the alternative is to re-wire their entire new catalog of software); under further use testing and scrutiny in Denmark, its repudiation of the basic intentions of XML will be highlighted; and under scrutiny in Denmark the thorough dependency of MSOOXML documents upon the Microsoft stack (Vista, Exchange, Sharepoint, Outlook, MS SQL Server, IE7, Office 2007, Groove, etc.) and their lack of interoperability & compatibility outside the new Microsoft stack will be underscored and well understood. The Danes will find that MSOOXML is no solution.

Regarding document interoperability the Danes will learn on their own about the three possible solutions, two of which are free and one still requiring funding...

1) the Microsoft-Clever Age-Novell "MCAN" Translator (at sourceforge and being integrated into only Novell's version of OpenOffice.org)

2) the Sun Microsystems Plugin (still in development and promising document exchange fidelity equivalent to OpenOffice.org, which Massachusetts originally deemed inadequate for its decentralized migration)

ODF Alliance membership passed 300 in recent months, not 9 months from the start of the lobbying and communications group. An analysis of the mix would indicate the proportion of ICT vendors, professional services providers, software development groups and end user agencies and organizations.

These are people with a view toward supporting ODF in software or establishing ODF policies or moving forward on migration. They may be entities who are already using OpenOffice.org -- and thereby ODF -- already. They are united by a confidence in a singular idea: common sense will prevail in the electronic document standards fight.

There are 50 countries represented in the list (and all continents but Antarctica).

I attended the Harvard Berkman Center discussion on Friday with the usual ODF suspects -- who it is always good to see...like following Phish -- and with the addition this time of a whole host of interested parties from the Consumer side from places like Norway, Denmark, Belgium, Yale. You know, foreign places, where in most cases the commitment to the concept of a universal, portable document format is more palpable than in the pecuniary, milque-toast USA.

The conference was assembled with skill by CPTech together with the TransAtlantic Consumer Dialogue (TACD). And it produced this extraordinary entry by Jamie Love on Huffington Post: "When Standards Are Political -- ODF". This is a high-water mark for speaking normal to the normal folks.

During the day full up with panel discussions, it was Jamie who prostrated himself with the dumbest of dumb questions in order to bring the conversation to a level that at least XML experts could understand. I'm sorry to say we geeks didn't take the hint and continued for most of the morning to ramble on quite beside the basic points that would orient people who don't spend all day steeped in ODF. (I know Jamie knows ODF better than his questions indicated, and his article proves it.) Only in the afternoon did the ole policy veterans from the EU inject a bit of sanity and common sense.

It was a long day, to be honest. But I still want more of these.

Why?

Why? Because the consumer groups and the highly intelligent and well-networked people there are key to getting the ODF story out on multiple fronts, on all of which we need to make our seven impressions...and then, btw, deliver better software.

The best & brightest in that room including Jamie and Laura DeNardis (of the Information Society Project at Yale Law School) are headed to Athens, Greece, next month to the Internet Governance Forum which comes under the auspices of the UN. There, at the IGF, there will be a proposal for global norms to support open standards for key aspects of info-tech -- including file formats.

Jamie intends to propose a sort of multi-national declaration of policy intent toward a universal file format standard (like ODF). Wish him luck, since we can use PR like that...as well as a few more policy adoption cases. How about Argentina? Canada? UK schools? South Africa? Singapore? Jeff -- how about another saki-bet?

Anyone serious about affecting balance in the software markets in the last ten years has had to address the need for another office suite to compete with Microsoft's Office franchise. Anyone, by turns, who is really serious in this objective has naturally and rightly gone one step further, to address the need for another format for documents to compete with Microsoft's .doc, .xls, .ppt and other formats.

Sun Microsystems rose to the challenge in acquring Star Division and in (almost) open sourcing the code to OpenOffice (the project has never been open enough to solve some of the more vexing technical problems of document interoperability). And everyone who has pitched in on OpenOffice, for better or worse between 1999 and the present, has risen in their own modest way to the call. Similarly true of KOffice and, now, of the AJAX-driven office suite services.

In the last few years the battleground, or rather the emphasis, has shifted away from the software applications themselves to where it should be. The XML-based file format originated in the early versions of OpenOffice/StarOffice got a name change -- to "OpenDocument" -- and got into the ISO ratification process in May 2005 and was ratified as an ISO standard a year later. ISO ratification generated a surge of interest in the format, fresh belief that an alternative could exist, and kicked off IT policy conversations in government offices around the world. These conversations have borne fruit in places like Massachusetts, Minnesota, Denmark, France, Belgium, the United Kingdom, South Korea, Malaysia, Norway, Germany -- to name the ones who have permitted their processes to be public -- and such conversations have successfully and naturally drifted into actual policies as well as plans for implementation of ODF where it counts on the desktops PCs throughout the offices of governmental agencies.

These are extremely positive developments for renewing competive balance in software markets, particularly since a year ago not a soul predicted such a level of rising interest in something as arcane as a document file format standard. And the trend is more likely to expand than contract -- even despite the possibility that Microsoft's upcoming XML-based file formats may get ISO approval, too. (IBM's Bob Sutor -- a mathematician by background -- said deep in his blog somewhere that the Standard Deviation of our year-ago expectations for ODF versus reality has been exceedingly wide; and it will likely to continue to widen even as we ratchet up our expectations.)

Looking back on where we were, where we've been, what we thought and said, I'm still surprised that it has taken so long for the emphasis to shift to the file format -- where the market control-points actually reside. However, I take solace in the reminder that we are talking about real Diffusion of Innovation. If we consult Rogers, Moore & Christensen on the topic, then we recognize that we are in the midst of a historical shift in the way people do things. Populations, particularly large populations in the hundreds of millions, take time to absorb new methods, new habits, new customs and new ideas.

Looking back, as I was doing, I found an article from Spring 2002 by Aaron Rouse in the Inquirer; that online journal was early to recognize the meaning of OpenOffice and of what OpenOffice and its format meant. (The OpenOffice 1.0 launch came only two months after this article, which you might say was three or four years ahead of its time.)

The Rouse article is really worth reading because even in 2002 Mr Rouse was sensitive to the importance of the file format, and the article walks the reader briefly through his experiences of particular software dependencies which drove his offices in certain directions. Business processes, he said in 2002, are at the heart of how Microsoft engineered a form of behavioral control over the wide market. And it will serve us well today to recognize how the mechanism works if we are going to unwind this particular Equilibrium.

The nexus of competition in the software markets is the common document format. Eyes are on the prize.

The headlines should read instead, "Microsoft Supports Self with ODF".

The entire press of the world -- including CNet, ZDNet, Financial Times, The Wall Street Journal and others -- have consumed this whole, almost at face value. It demonstrates at once how obscure the file format topic is and how desperate the press are to read into the scene now another ODF win. But they're getting the real and the visionary all twisted up in a bunch. Even technology analysts who have demonstrated a basic understanding of file formats in the past are getting this quite turned around.

Everyone -- to a man -- is saying that Microsoft has caved in to ODF. It's simply not the case if anyone examines the facts.

The Microsoft press release talks about this as "...developing bidirectional translation tools..." If anyone takes the time to go to SourceForge, where this project is posted for development and download, they will notice the project consists of a single Office add-on for Word 2007, a product that does not yet exist on the market. They will further notice that the tool -- singular, so far -- is in fact uni-directional.

Open XML Translator provides tools to build a technical bridge between the Open XML Formats and Open Document Format(ODF). As the first component of this initiative, the ODF Add-in for Microsoft Word 2007 allows to Open & Save ODF documents in Word. [emphasis added]

And scratching the surface a bit more, the curious observer will note that the tool consists of a few eXtensible Stylesheet Language Transformation (XSLT) scripts which change ODF's open XML content into MSECMAXML's corrupted, tokenized, non-expressive & non-human-readable XML. That means Office 2007 -- a product not in circulation -- will one day be endowed with the ability to open an ODF file and save it as the Microsoft "Open XML" format. XSLT is nothing special: it is a common way to transform data in ODF to just about any other format, including HTML, XHTML, DocBook, what have you.

The translator project does not yet EXPORT to ODF, demanding a shaded interpretation of "interoperability" here. What's more, questions will persist if this "open source" project will ever be open in spirit (that is, attended by individuals driven by practical self-interest)...or interoperable. Chances are, not, if we interpolate from past Microsoft behavior (Microsoft's contrived rejection of PDF export | Groklaw). Eyes of the cognoscenti are peeled.

It takes pause. Think a minute. ODF files are being shifted into MSECMAXML, possibly the most proprietary format ever proposed in the history of standards. MSECMAXML is a private implementation of open XML so clogged with binary flotsam & jetsam, to which only Microsoft customers will ever gain access, that the attempt to pass the translator off as a good thing should raise further questions about Microsoft's intentions.

Through the obscurity of arcane, black-art file format techniques, Microsoft have made something beneath trivial into a revelation of interoperability with the intention to freeze the balance of the global public sector from following in the footsteps of Massachusetts, Munich, France, Denmark & Brussels. It's a skillful (even creative) bit of PR but in software terms, a cypher. So powerful is the ODF concept to the business of government that I don't believe California, Minnesota, Bristol, Podgorica, Rome, Kuala Lampur, Tokyo, or even Beijing would be deterred by this whafer-thin gossamer of FUD.

Revealed then is Microsoft's intention to avoid -- it seems at any cost -- actively supporting the OpenDocument Format in its software. This is the true message to take home.

Bob Sutor on Thursday compiled a useful mix of early reactions which were each mistaken in tone, emphasis, proportion and fact.