Microsoft and ODF: Has Hades Gone Sub-Zero?

Most of the time, Microsoft's public declarations are pretty easy to parse. A bit of pre-announcement here, a touch of FUD there, with the odd dollop of feel-good waffle thrown in for good measure. Occasionally, though, it produces what can only be called a googly – not to be confused with a Google – with announcements like this one about adding support for ODF in Microsoft Office:

When using [Microsoft Office 2007] SP2, customers will be able to open, edit and save documents using ODF and save documents into the XPS and PDF fixed formats from directly within the application without having to install any other code. It will also allow customers to set ODF as the default file format for Office 2007.

At first sight this seems incredible – a complete U-turn from Microsoft. But it's important to remember that we have been here before:

Expanding on its customer-focused commitment to interoperability, Microsoft Corp. today announced the creation of the Open XML Translator project. The project, developed with partners, will create tools to build a technical bridge between the Microsoft Office Open XML Formats and the OpenDocument Format (ODF). This work is in response to government requests for interoperability with ODF because they work with constituent groups that use that format. In addition to being made available as free, downloadable add-ins for several older versions of the Microsoft Office system, the translation tools will be developed and licensed as open source software. The translation tools will be broadly available to the industry for use with other individual or commercial projects to accelerate document interoperability and expand customer choice between Open XML and other technologies.

As we now know, those “translators” were pretty useless, and so one concern has to be that Microsoft's ODF support in Office will be less than perfect. But even if they are, the signal that Microsoft is sending to users is still extremely strong: that ODF is supported, and that if you use Microsoft Office, you can happily adopt ODF as the default format. If it turns out that these statements aren't true, then customers will be unhappy, and the blame will lie squarely with Microsoft. So I don't think the imperfect support argument is very strong.

What about this, then?

Microsoft will join the Organization for the Advancement of Structured Information Standards (OASIS) technical committee working on the next version of ODF and will take part in the ISO/IEC working group being formed to work on ODF maintenance.

As I've written elsewhere, I see increasing signs of new Microsoft approach to open source, which involves loving applications to death, while undermining GNU/Linux. The idea might be to lull the wider free software community into a false sense of security while digging away at the foundations, so that one day open sources apps find themselves running mostly on Windows, with Microsoft in the driving seat.

That's more of a long-term threat, albeit one that the free software world needs to be aware of. So, just for the sake of argument, let's assume that Microsoft is sincere, that it really will offer proper ODF support in Office, and that it really wants work with rather than against the OASIS technical committee: why might that be?

I'm sure that one reason it feels able to make this move is that as Matt Asay astutely points out, for the corporate sector, the game has moved on:

People are agog that Microsoft has announced support for Open Document Format (ODF), but I'm not sure why. This was a foregone conclusion once Microsoft figured out how to move lock-in above the file level to the content network.

In other words, to Sharepoint.

Microsoft has been hell-bent on getting enterprises to dump content into its proprietary Sharepoint repository, calling it the next Windows operating system. I call it the future of Microsoft lock-in.

Microsoft doesn't need to zealously guard file formats anymore. It already owns the next few decades of lock-in, and many enterprises are willy-nilly dumping their content into Microsoft's proprietary repository at a pace and in a manner that is as potentially destructive for those enterprises as it is beneficial to Microsoft's income.

Matt is absolutely right to finger SharePoint as one of the least-appreciated threats to open source. But I also think other important factors were at play in this latest decision – after all, not everyone uses SharePoint, and so there is still a considerable downside for Microsoft.

I think the answer in part lies in the recent series of defeats that Microsoft has suffered at the hands of the European Commission. These have been about interoperability, or the lack of it, and there are other investigations underway that also involve this issue. Maybe the the fines that the company has had to pay have finally reached the level – one involving nine zeroes - where even Microsoft is forced to acknowledge them and do something about it. Adding proper ODF support should head off much of the criticism it is facing in Europe.

Finally, it's also interesting to note that criticism of Microsoft's file formats was expressed in a rather different way recently, during the bitterly-contested ISO vote on OOXML. Although Microsoft's machinations mean that it looks like OOXML will be approved, the voting pattern delivered an undeniable slap in the face for the company, as others have noted:

In an ominous sign for Microsoft (NSDQ: MSFT)'s growth prospects in emerging regions, countries that represent the world's fastest growing tech markets voted against accepting the company's latest Microsoft Office document format as an international standard.

Brazil, India, and China, which together count for more than a third of the world's population, all voted against Office Open XML in voting last week before the International Organization for Standardization (ISO). Russia was the only member of the so-called BRIC nations to vote in favor of ISO ratification for OOXML.

As Microsoft well knows, these markets are where most of the future growth can be expected. If they are bent on ODF adoption regardless of ISO ratification for OOXML, Microsoft will effectively be shut out of the hottest markets unless it builds some bridges (one of its favourite metaphors at the moment).

Supporting this view is the fact that Microsoft's latest announcement also includes news support for the less well-known (in the West, at least) Chinese national document file format standard, Uniform Office Format (UOF):

Microsoft is also committed to providing Office customers with the ability to open, edit and save documents in the Chinese national document file format standard, Uniform Office Format (UOF). The company does so today by supporting the continued development of the UOF-Open XML translator project on SourceForge.net, and will take additional steps to promote the distribution and ease of use of the translator. As UOF develops and achieves market adoption in China, Microsoft will distribute support for this format with Office to its customers in China.

I seriously think Microsoft is up to something here. I mean come on, it MICROSOFT, the people who want their crapware installed on every computer under the sun, the people who are continuously putting Linux down. From the above article, I have concluded that Microsoft is most likely trying to sneak up on open source software in general, for the soul purpose of killing it. I believe in about 10 to 20 years from now, Microsoft will attempt to take out the GPL, which may result in the largest international software monopoly ever conceived. The best thing for everyone to do is just IGNORE Microsoft completely. If enough people ignore such a horrible corporation, then it's customer base will start shrinking, eventually resulting in bankruptcy, which where Microsoft deserves to be for it's past actions.

don't know the full story of the OOXML saga but...
supporting ODF natively makes releasing and solving all the dark spots in OOXML less stringent
maybe this way they can avoid opening up office 2007 native format and keep their secrets to themselves?

I tried the add-ons for Office 2003 from both MS and Sun and found the Sun product functioned vastly better. Not only did one need to apply separate add-ons from MS for each Office app, but numerous discrepancies (errors) existed in the import of ODFs. The Sun add-on was a single installation for all Office apps and it worked perfectly. All docs I tested displayed exactly as they did in native ODF apps like OOo. I guess that's not surprising given Sun ODF experience, but with an open format (ODF), it's amusing to see MS have more trouble writing a good filter for it's own Office product than a competitor.

Bobber47 said:
"it's amusing to see MS have more trouble writing a good filter for it's own Office product than a competitor."

Well, it's not too hard to see that Microsoft probably intends to blunt adoption of ODF by offering inferior ODF tools while touting their own native file format. OOXML in Office probably never will be the ISO version. Likely scenario: Migrate from ODF to OOXML - the new standard! Who if anybody is actually going to go look up the multipage standard?

They get to say: We support ODF.
Customer realizes: ODF support is difficult to implement while OOXML is built in and bulletproof.
Result: Customer uses OOXML.

This sounds quite familiar to MS antics against Netscape.
Javascript was adopted as an official international standard, and Microsoft jumped on board. Only it was more an attempt to capsize Javascript....

Netscape had many major improvements and extensions that they wanted to see, but Microsoft flooded the sessions with attendees that prevented forward progress, all the while pursuing their own JScript development to dethrone Javascript.

Pretend for a moment that you are being chased by a huge, rabid angry dog. After you've ran for as long as you can, you finally stop and face the monster. It quickly reaches you, but instead of attaching, it sets an ice cold beer at your feet. My reaction to this news was exactly the same as it would be in this example.

Also, I do worry about their joining OASIS. "Embrace and extend" anyone? Perhaps I'm just too paranoid, but old habits die hard.

I think Microsoft is just covering their bases. Now they'll be the only office suite that supports both ODF and OOXML (now both ISO standards) and I think that they'll use it as selling point. Since OOffice will never support OOXML due to ahem, questionable documentation, this, along with a pile of used $20 bills, should be enough to keep most governments from going open source.

I'm more interested in how is M$ going to handle China, which seems determined to have full control of their government software and not have the threat of an NSA back door looming over it. This means going open source and that's a huge market to support further open source development.

"... this, along with a pile of used $20 bills, should be enough to keep most governments from going open source."

I think that once people/governments/businesses start using odf they will start using the software that is best suited for the job. In a gobalised market businesses and gov's need to be able to communicate efficiently. Based on the idea that emerging markets are turning to opensource, the switch by older "mature" markets to opensource seems inevitable. It's business.

I think MS had to this as it was requested by the customers. Sure OOXML is "open" but maybe people asked for ODF support. An when large companies and government tell you they want ODF or else you really have to listen even if you're MS.
Office is the best at what it does , no doubt about that and people , companies are used to it , rely on it so as long as Office sells I'm sure MS won't have a problem with supporting ODF. They probably just feel they can compete with Open Office even without any file format "tricks" and they're probably right.
Don't get me wrong, I'm no MS fan and I use Open Office but clients who spend the big bucks on Office now will continue to do so in the future and OO will probably only attract a small and insignificant revenue wise client base.
Once Office will be able to save and open ODF files then ? Then what ? It will be business as usual and one more feature for Office.

I think MS had to this as it was requested by the customers. Sure OOXML is "open" but maybe people asked for ODF support. An when large companies and government tell you they want ODF or else you really have to listen even if you're MS.
Office is the best at what it does , no doubt about that and people , companies are used to it , rely on it so as long as Office sells I'm sure MS won't have a problem with supporting ODF. They probably just feel they can compete with Open Office even without any file format "tricks" and they're probably right.
Don't get me wrong, I'm no MS fan and I use Open Office but clients who spend the big bucks on Office now will continue to do so in the future and OO will probably only attract a small and insignificant revenue wise client base.
Once Office will be able to save and open ODF files then ? Then what ? It will be business as usual and one more feature for Office.

As someone who deals with businesses, I have found the reason they stick with Office is that many of their documents have embedded macros.

These perform useful tasks like requesting specific information in Word, creating report panels in Excel, and making entry forms and reports in Access. In most cases these were written by long since gone staff.

Even though OpenOffice and Google docs are more cost effective solutions and often used where general work is performed, the costs of migrating Office macros, reports and input forms is more than the cost of relicensing.
In many cases. 3rd party Office support will migrate existing macros etc to new versions of Office as part of their support package, but apart from Linux vendors like RedHat and Novell, a service to migrate Office macros, etc, to OO formats are not available for the free version of OpenOffice, or in cases where I've seen it offered are stupidly expensive.

I have often wished I could write a program to do this work, especially for database input forms and reports, as it would make my life easier and get rid of the most significant hurdle in adopting OO and the AMP stack. Maybe now that SUN owns MySQL and OpenOffice ....

I'm not sure that Windows will continue or even be necessary in light of the browser becoming the platform of choice for Web2, but Office with its installed base is where MS is drawing the line in the sand for its new web based business model.

Maybe I've grown suspicious by nature, but when I read that Microsoft is joining OASIS I immediately thought about the ISO mess. Has MS found similar weaknesses in OASIS's rules that would allow it to sign up a bunch of its partners and effectively take over the direction of ODF, perhaps?

I take the gimp format. This doesn't map to ODF. gimp has room for (i think)... layers, session data (where I left my placed window on shutdown), perhaps plugins active and their state at shutdown (or let's pretend), undo history,.. and various other things.

I want to create office documents that integrate well with gimp. I can do that in the FOSS world, but let's say I want to be able to take a snapshot of the result.. in other words, I want to persist the entire state of everything to disk.. in other words, I want a file format that boils down to odf+gimp format but integrated.

Now repeat with every other app you like (3d modeling, animation, music, map software, python scripts, special kernel state, special x windows state, etc).

In fact, think of a file format that snapshots anything you want from the platform.

OK. This is what Monopolysoft will be able to do and label it ooxml or whatever.

In our FOSS world, these things I mentioned map to numerous files, directories, config info, and maybe auto generated utility apps, databases, etc, but the mix is not standardized. So, for example, two different FOSS-based third party contract jobs to mix these apps to solve a business problem for someone will come up with different solutions that won't interoperate. Specifically, we certainly don't imagine that everything from either of these jobs can be persisted to fit into ODF in a well-defined fashion. It won't fit into ODF period unless we do things like [paragraph id="gimp-info-25262435"]... {gimp format plus mixture coding using some home-brewed integration semantics} ...[/paragraph]

Do you see where I am going? Monopolysoft can always come up with gunk that will not map to ODF cleanly, that will not have all semantics fall neatly into what is defined in ODF, eg, for the standard tags and attributes. Hence, they can abide by ODF but include all the other gunk as binary blobs intermixed in their own way with the expected normal user content data.

..This is called "extend".

In fact, not extending ODF would be boring, for example, macros are currently not supported (ie, standardized in every facet), but they are very real, are used, are saved, and have real semantics that Openoffice will abide by. However, prior to standardization, KOffice may not do the Right Thing with the same macro content (unless the two projects are working closely together and maybe using the same macro processing libraries).

Thus, almost any practical standard can be extended legally: just add feature gunk, stir, and persist into the payload space of the target standard. ["payload" is what I call the data area that is not really restricted; that's opaque or just not well-defined.. ie, where the user content goes.]

And the question is not "is there a way to do something in a standard fashion," because the state of software analysis today is such that there is a lot that is not (in a practical fashion) provably correct or incorrect. Monopolysoft can just claim that their new wowser feature doesn't map to the standard tags or standard macros or Java or python or anything else. For example, if we are talking about functionality implemented in dotregret, there may not exist a faithful mapping to a standard feature of ODF (certainly not one to work on every platform 100% identically and that can be applied automatically or in real time).

***** Mini aside *****
I don't see a solution between Monopolyware and the rest of the world.

In fact, generally speaking, we'd need open source in order to interoperate if we want anything but a simple and static feature list. That is why almost(?) every closed source category of software is owned by a single company (a mini monopolist), at least to the extent that interoperability would be expected and the feature list grows.

So to have multiple vendors in a space where we expect interoperability and that the vendors will add features ahead of the standard, we'd probably have to force the vendors to stick to FOSS 100%. In practice, we can allow some closed source, but it should not be for anything "important." I don't know how to define "important", btw.

I also think Monopolysofts' monopolies put them in a place of their own among closed source vendors. I wouldn't use their closed products at all if I wanted interop.
***** end *****

Of course, Monopolysoft has in the past broken the actual standard to even more easily thward competitors once they owned much of the market. But with authorities and customers watching, they'd want to avoid doing that.

Here is where I think OOXML comes in. It's Broken By Design and in a way that Monopolysoft crafted.

If they extend ODF as above in order to get a full MSOffice experience inside ODF, it won't be easy to hide. People will demand more information since it will be obvious that MSOffice will not behave like Openoffice with the same MSO file. At that point, we can have Monopolysoft explain what is going on and then we add the extensions to ODF in a clean way (but they may get away with their lock-in during this time as people snap up their products and if authorities sit back).

They won't want to be so helpful, however. They might do this: point to OOXML and say that they are just mapping OOXML to some binary stuff that they had to put inside ODF somehow (without violating the std). They'll say that OOXML is an ISO standard and that we should just use the bridge MS created and go on with our merry lives effectively following OOXML for the extensions.

Broken By Design OOXML means that there will be much confusion and lack of interoperability among all third parties actually trying to implement it. Also, there is enough in there and crafted in such an interdependent complex way to hide tricks more easily than with ODF. [not to mention that it is tuned to Monopolysoft's factory]

We might want to do this in response to the expected Monopolysoft excuses and OOXML suggestions: declare MSOffice with extras as unfit once we reach a point where we see the loads of gunk that cannot be explained easily and cleanly. Monopolysoft pointing to OOXML as the answer should result in an automatic disqualification of MSOffice since OOXML is not fit and is not accepted (assuming the standard accepted was ODF and not OOXML).

Thus to get new sales, Monopolysoft would have to restrict MSOffice to ODF basics or come clean in a sound way about their extra feature encodings (ie, the "gimp"-ish add-ons).

But how could be measure this extra stuff officially if Monopolysoft continues to stack ISO and now OASIS? It's tough to appeal to the ODF format if that itself is being corrupted or changed to allow almost anything.

We will thus have to have requirements on the Monopolist that are set by some group not ISO (unless ISO is cleaned out).

Anyway, I am imagining what some government buyers might want to do. Personally, I would ban closed Monopolyware, for security and audit reasons. End of story.

Others may want to point to existing EU unfulfilled requirements and insist that a prerequisite to doing business be that the EU give a clean bill of health.

I take the gimp format. This doesn't map to ODF. gimp has room for (i think)... layers, session data (where I left my placed window on shutdown), perhaps plugins active and their state at shutdown (or let's pretend), undo history,.. and various other things.

I want to create office documents that integrate well with gimp. I can do that in the FOSS world, but let's say I want to be able to take a snapshot of the result.. in other words, I want to persist the entire state of everything to disk.. in other words, I want a file format that boils down to odf+gimp format but integrated.

Now repeat with every other app you like (3d modeling, animation, music, map software, python scripts, special kernel state, special x windows state, etc).

In fact, think of a file format that snapshots anything you want from the platform.

OK. This is what Monopolysoft will be able to do and label it ooxml or whatever.

In our FOSS world, these things I mentioned map to numerous files, directories, config info, and maybe auto generated utility apps, databases, etc, but the mix is not standardized. So, for example, two different FOSS-based third party contract jobs to mix these apps to solve a business problem for someone will come up with different solutions that won't interoperate. Specifically, we certainly don't imagine that everything from either of these jobs can be persisted to fit into ODF in a well-defined fashion. It won't fit into ODF period unless we do things like [paragraph id="gimp-info-25262435"]... {gimp format plus mixture coding using some home-brewed integration semantics} ...[/paragraph]

Do you see where I am going? Monopolysoft can always come up with gunk that will not map to ODF cleanly, that will not have all semantics fall neatly into what is defined in ODF, eg, for the standard tags and attributes. Hence, they can abide by ODF but include all the other gunk as binary blobs intermixed in their own way with the expected normal user content data.

..This is called "extend".

In fact, not extending ODF would be boring, for example, macros are currently not supported (ie, standardized in every facet), but they are very real, are used, are saved, and have real semantics that Openoffice will abide by. However, prior to standardization, KOffice may not do the Right Thing with the same macro content (unless the two projects are working closely together and maybe using the same macro processing libraries).

Thus, almost any practical standard can be extended legally: just add feature gunk, stir, and persist into the payload space of the target standard. ["payload" is what I call the data area that is not really restricted; that's opaque or just not well-defined.. ie, where the user content goes.]

And the question is not "is there a way to do something in a standard fashion," because the state of software analysis today is such that there is a lot that is not (in a practical fashion) provably correct or incorrect. Monopolysoft can just claim that their new wowser feature doesn't map to the standard tags or standard macros or Java or python or anything else. For example, if we are talking about functionality implemented in dotregret, there may not exist a faithful mapping to a standard feature of ODF (certainly not one to work on every platform 100% identically and that can be applied automatically or in real time).

***** Mini aside *****
I don't see a solution between Monopolyware and the rest of the world.

In fact, generally speaking, we'd need open source in order to interoperate if we want anything but a simple and static feature list. That is why almost(?) every closed source category of software is owned by a single company (a mini monopolist), at least to the extent that interoperability would be expected and the feature list grows.

So to have multiple vendors in a space where we expect interoperability and that the vendors will add features ahead of the standard, we'd probably have to force the vendors to stick to FOSS 100%. In practice, we can allow some closed source, but it should not be for anything "important." I don't know how to define "important", btw.

I also think Monopolysofts' monopolies put them in a place of their own among closed source vendors. I wouldn't use their closed products at all if I wanted interop.
***** end *****

Of course, Monopolysoft has in the past broken the actual standard to even more easily thward competitors once they owned much of the market. But with authorities and customers watching, they'd want to avoid doing that.

Here is where I think OOXML comes in. It's Broken By Design and in a way that Monopolysoft crafted.

If they extend ODF as above in order to get a full MSOffice experience inside ODF, it won't be easy to hide. People will demand more information since it will be obvious that MSOffice will not behave like Openoffice with the same MSO file. At that point, we can have Monopolysoft explain what is going on and then we add the extensions to ODF in a clean way (but they may get away with their lock-in during this time as people snap up their products and if authorities sit back).

They won't want to be so helpful, however. They might do this: point to OOXML and say that they are just mapping OOXML to some binary stuff that they had to put inside ODF somehow (without violating the std). They'll say that OOXML is an ISO standard and that we should just use the bridge MS created and go on with our merry lives effectively following OOXML for the extensions.

Broken By Design OOXML means that there will be much confusion and lack of interoperability among all third parties actually trying to implement it. Also, there is enough in there and crafted in such an interdependent complex way to hide tricks more easily than with ODF. [not to mention that it is tuned to Monopolysoft's factory]

We might want to do this in response to the expected Monopolysoft excuses and OOXML suggestions: declare MSOffice with extras as unfit once we reach a point where we see the loads of gunk that cannot be explained easily and cleanly. Monopolysoft pointing to OOXML as the answer should result in an automatic disqualification of MSOffice since OOXML is not fit and is not accepted (assuming the standard accepted was ODF and not OOXML).

Thus to get new sales, Monopolysoft would have to restrict MSOffice to ODF basics or come clean in a sound way about their extra feature encodings (ie, the "gimp"-ish add-ons).

But how could be measure this extra stuff officially if Monopolysoft continues to stack ISO and now OASIS? It's tough to appeal to the ODF format if that itself is being corrupted or changed to allow almost anything.

We will thus have to have requirements on the Monopolist that are set by some group not ISO (unless ISO is cleaned out).

Anyway, I am imagining what some government buyers might want to do. Personally, I would ban closed Monopolyware, for security and audit reasons. End of story.

Others may want to point to existing EU unfulfilled requirements and insist that a prerequisite to doing business be that the EU give a clean bill of health.

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.