Posted
by
Soulskillon Wednesday July 23, 2014 @12:17AM
from the won-the-battle,-working-on-the-war dept.

Andy Updegrove writes: "The U.K. Cabinet Office accomplished today what the Commonwealth of Massachusetts set out (unsuccessfully) to achieve ten years ago: it formally required compliance with the Open Document Format (ODF) by software to be purchased in the future across all government bodies. Compliance with any of the existing versions of OOXML, the competing document format championed by Microsoft, is neither required nor relevant. The announcement was made today by The Minister for the Cabinet Office, Francis Maude. Henceforth, ODF compliance will be required for documents intended to be shared or subject to collaboration. PDF/A or HTML compliance will be required for viewable government documents. The decision follows a long process that invited, and received, very extensive public input – over 500 comments in all."

At the end of the day, you need to get stuff done. If the propriety vendor got a monopoly on the easiest/fastest/most convenient ways of doing things, then it would be wasteful to spend time/money on ways to resist it. This is a case for Microsoft Office before the triumph of various other office packages that came along. When majority of your correspondents use Office, why would you spend the extra time making your documents in something else

"If you are ready to pay". There you go. I *already* paid: Any document which is produced by a government official was paid for *by me*, in the form of my tax money. I would expect to be able to read these documents without additional charge. If a *company* decides to go for vendor lock-in, that's their business - they should be able to do the "easiest/fastest/most convenient" calculation themselves. If it turns out they can't read their old design documents anymore, they have the right to pay a team of engineers a lot of money to reverse-engineer their old stuff. They will factor in these costs in their next product, and I have the choice to buy it, or shop elsewhere. However, this is not the case for the government. I cannot simply "shop elsewhere", so I expect the government not to cut corners and factor in what's easiest/cheapest/most convenient for their citizens.

When iWork first shipped, I asked folks in the know (at Apple) why they chose to design/engineer a completely new suite of file formats rather than adopting/utilizing ODF. I was told it was because ODF wasn't mature enough for their needs, and that it was felt that the ODF working group would be too slow for the iWork development roadmap.

So far, ODF has chugged along, consistently; while iWork has seen a divergence in format compatibility (between Mac and iOS versions) and a complete, from-scratch rewrite (in the most recent version) that torpedoed backwards compatibility.

Enough is enough. If Apple would have embraced ODF, they'd have rocketed the world's move away from Microsoft's Office document stranglehold. Instead, they have squandered both an opportunity to further stomp a odious competitor as well as an opportunity to position their desktop and mobile products as the best commercial competitor for the future where ODF clearly will reign supreme, all in one stupid "Not Invented Here" design decision.

Good decition from UK. But one has to ask why not ten years ago. And why not in all countries. Instead MS has been allowed to nominate it's own closed format as open standard! And continue ruling and taxing the globe. And making competition impossible.And yes, ODF is not perfect. Nothing is. And ODF will continue to evolve like any format. The key is that it is open and allows (opens) competition.

I really hope this catches on with businesses as well. I'm writing a lot of job applications at the moment, and being financially challenged I'm doing the work from LIbre Office. If I convert my application and CV to.doc or.docx the formatting will be all wrong when a potential employer reads it. Therefor I've been converting everything to PDF before sending. I'm starting to see job ads now that actually require people to deliver in PDF, most likely for the same exact reason, but I'm not entirely sure everyone can figure out how to convert a doc/docx/odf to PDF.

There are a lot of people out there with very limited computer skills, so I think a well supported open document standard will be good for everyone in the long run.

Where I live (in the Netherlands), there is even a university that requires job applications to be submitted in either.doc or.docx. So, my nicely formatted.pdf (pdfLaTeX) goes into ImageMagick where it is converted to a sersies of.png. That is imported into LibroOffice and stored as.doc/x. Swallow that. Now, the files I submit are "only" 7.5 times bigger then necessary. Text cannot be selected anymore (by them), but that's their problem. And it didn't hurt: I was selected for intervi

If I convert my application and CV to.doc or.docx the formatting will be all wrong...

Did you actually try that? On windows you can use free.doc viewer [microsoft.com] to verify how will word render your document. I believe that the mess will be comparable to mess created just by using a different version of MS office. I.e. not significant.

I believe that the mess will be comparable to mess created just by using a different version of MS office. I.e. not significant.

It really depends. If your resume/cv is a basic, barely formatted text document, sure. Nothing bad will happen. But I've been going back and forth between Word and OpenOffice/LibreOffice for over a decade now at various times, and any document with complex formatting and precise spacing (as is often true with resumes/cvs) is bound to be a disaster when viewed in Word. All the formatting will basically convert "correctly," but the spacing and rendering are often way off, making the document look like you

All the formatting will basically convert "correctly," but the spacing and rendering are often way off, making the document look like you didn't pay enough attention to detail (exactly the OPPOSITE of what you want in a job application).

Sometimes that is a typeface issue. The fix is to make sure both machines have the exact same ones available.

If your job requires you to send a resume electronically, you should have the skills to create a PDF; they kind of go hand in hand. It's actually a clever sort device on the part of HR. (Not that I'm accusing HR of ever being clever).

Assuming there's compliance with this edict at some point in the foreseeable future (which is questionable); what's going to happen is that people will save as ODF from Word. The question is then whether you can truly use other software to work on those documents. MS has a long history of failing to properly implement standards; or even their own specifications.

I suppose that doesn't quite answer the question of whether any of those pieces of software actually produce documents that comply with the standard, which is what the actual requirement is.

But in any case the practical difficulty will be that people will say they have complied with the requirement by saving as ODF from Word, whether or not that actually enables interoperability with non-Word software.

We (the UK) are about to embark on another round of austerity, regardless of who wins the next election. I'd like to see what the public thinks about mass conversions of Word/ Excel/ PP docs - because it's not going to be quick or free, and once we reach the stage of 'well, what benefit will this give us right now?', there isn't one - in fact, it's the opposite.

If the cabinet office wanted to do this with purely internal documents, they might have a chance - but if any docs come in or go out of the office,

As a UK tax payer, I welcome the move. Finally, someone in government is looking further ahead than just the next election.

I would imagine that someone at GCHQ could easily convert the documents for a tiny fraction of the budget that they've got. In fact, they've probably already got conversions of everyone's private/secret documents already.

Plenty of money for spying on UK subjects, but no money for protecting their interests in not being tied to a predatory US company.

The obvious tangible benefits are that the documents will no longer be locked into some stupid proprietary format that can never be converted due to ridiculous macros and scripts (quite why a static document needs to have macros and scripts is beyond me).

Your argument about not wanting to change something due to the length of time that it's been unchanged is laughable. I don't get why you are so opposed to such a sensible, intelligent change. I bet you could have used the self-same arguments for keeping government documents on micro-fiches (or similar old-time technology).

Short term it may cost more, long term it should save a lot... As someone who fully expects to still be paying taxes in 10 years time, i welcome long term savings.

As for interoperability, they are the government... You either want their business (eg suppliers), or you have no choice (eg taxpayers)... If they require that you submit documents in ODF then that's what you do, or they will find other suppliers who will.

And, if it's put to the public as '100 spying incidents on members of opposition parties or a document format that is completely free and belongs to the people', then I imagine you'll get a different response.

I'm curious, you seem to have a strong pro-microsoft agenda here. Why is that? (or are you just getting a bit old and are afraid of new-fangled technology)?

The vast majority of their users aren't especially smart when it comes to technology. They're essentially office workers - they don't give a stuff about the underlying format, they only care about being able to do their job.

The vast majority of their users aren't especially smart when it comes to technology. They're essentially office workers - they don't give a stuff about the underlying format, they only care about being able to do their job.

Not sure if you are saying that ODF is a good thing or a bad thing... I think the GP makes a great point. ODF as I understand it is an FOSS attempt to be like Microsoft Word, when as the GP suggests, we should transition to something human readable.

Why ODF? Because its the best format. It was designed very carefully by a very large team of stakeholders (including engineers, lawyers, document companies, the Vatican Library, Medical professionals, architects, electrical engineers, etc). It was reviewed and revised by large groups to ensure it would fit their needs. Its unencumbered by patents. NONE of this happened with microsoft's OOXML (as it is, there is no software that can read that standard, including no software from microsoft). Microsoft cannot support their own standard. Oh, and ODF is human readable.

That doesn't make much sense as this particular change would be welcomed by people who think that principles are important. In fact, a lot of changes are brought about by people who stick to their principles (e.g. abolition of slavery).

MS office is dying. how many people still have a non infected windows platforn that wasn't sitting behind a firewall set to block everything external? microsoft had a huge American base. they have failed so badly they had to cut 18k jobs with minimum 6 months of not having access before they can be rehired. microsoft isn't the only game in town, and the iphone and android platforms showed that people were ready for new tech, just not from microsoft. because they knew microsoft was screwing them over.

Exactly. The Office file formats are still extremely ubiquitous in the business world, and if you use something like LibreOffice to modify them, the formatting of the documents is almost guaranteed to go crazy, without you possibly not even knowing it, and ultimately losing customers. I'm sorry, but it has just been incredibly practical decision to just install the fucking Microsoft Office. Of course this UK Cabinet's decision is another step towards open standards, so I guess that's good.

if you use something like LibreOffice to modify them, the formatting of the documents is almost guaranteed to go crazy, without you possibly not even knowing it, and ultimately losing customers.

That is why you don't save in closed formats. If you open it in LibreOffice you save it in a format that is properly documented.My customers get a PDF. If they ask specifically for a proprietary format they get the proprietary format AND a PDF file. Not that I even use LibreOffice, I just have this feeling that Microsofts backwards compatibility is an ugly cludge that can break in corner cases between Office versions.As long as the customer gets the PDF they can see what the document is supposed to look lik

Many people work in situations where they must exchange documents with other people (inside or outside the company). When a document looks vastly different in LibreOffice compared to MS Office, that is a problem. At a previous job, I had to use Word on Windows -- Word on Mac was not enough -- when dealing with files containing MathType equations.

you get the same problem with people using different versions of Office, we've had countless problems with users who have the older versions of Word etc not being able to read the newer formats of DOCs, not only that but different versions of Word can format the same document differently

Yeah, this is because Office reuses so much of Windows, not just limited to basic API calls to get files and use control widgets and such, but rendering of fonts, etc.

LibreOffice has a much better chance of consistent document rendering on multiple platforms.

The Cabinet Office announcement does make a distinction between documents for collaboration and those for viewing ; PDF/A and HTML should at least have a reasonably consistent rendering (depending on how fancy you get with stylesheets in the case of HTM

When you are the little guy there is a lot of pressure on you to conform to the standards set by those you work with (and that may mean not just using MS office but using a specific version of MS office), when you are the big guy you SET the standards and require other people to conform to them.

bullshit. I get a.xls from my accountant to enter my details, and its full of protected cells and functions. I use LibreOffice, and so far my accountant hasn't even noticed anything untoward with the returned.xls file I send him.

Considering Word can't even open some Word documents created with older versions of Word, I think this is pretty damn good.

Why are you treating your customers as if they were your collaborators? They should never see your word processing documents, and they should never, ever, have access to your spreadsheets. Even in those situations where you have absolute confidence in the integrity and technical capability of your customer, you should not invite man-in-the-middle attacks with the inappropriate use of unsecurable formats.

Learn how to use PDF. Most current word processors and spreadsheets offer this as an export (I don't kno

Exactly. The Office file formats are still extremely ubiquitous in the business world, and if you use something like LibreOffice to modify them, the formatting of the documents is almost guaranteed to go crazy, without you possibly not even knowing it, and ultimately losing customers. I'm sorry, but it has just been incredibly practical decision to just install the fucking Microsoft Office. Of course this UK Cabinet's decision is another step towards open standards, so I guess that's good.

Outside of bookmarks in Writer I find OpenOffice/LibreOffice to generate more compliant documents to Microsoft's tools than Microsoft's tools do.
They're also typically smaller even when saved in the Microsoft formats.

I've actually been using ODF exclusively at the office (which uses entirely msft software) for every file I work on and email out, and nobody's noticed or complained yet. The standard install here is MessOffice 2010. Rather than try and change anyone else, I just took a look a the man in the mirror. It's gone well.

I do the same, though mostly as a small business. Very occasionally I find someone for whom the document doesn't work right. In those cases I simply say something like "oh; it must be a bug in MS Office; you might try LibreOffice; its available for free from https://www.libreoffice.org/ [libreoffice.org]" everybody I have done this for has downloaded that and been happy.

How well does word with odf actually work? does saving from word as odf and opening it in libreoffice have a higher or lower chance of screwing up the document than saving as doc and opening it in libreoffice?

I use ODF but no-one else does because MS Office doesn't properly support it, I'm crippling my ability to share documents around purely for ideological reasons.

Microsoft OSs are down to 14% market share.

It simply makes no sense to continue using their outdated lockin-inspired formats. The world needs to transition to document editing formats that're portable across whatever computing devices users want to buy.

ODF was designed by the Organization for the Advancement of Structured Information Standards (OASIS) consortium to be that set of formats in 2005, and was only derailed by an intense and deeply corrupt effort by Microsoft. It's incredibly sad that we've had to wait for almost a decade for governments to finally start the transition.

Disagree, at least from the points you make. The "best" format is the one that is the most useful to you, technical reasons be damned. If I use ODF but no-one else does because MS Office doesn't properly support it, I'm crippling my ability to share documents around purely for ideological reasons.

Well since the entire cabinet is now using it, they all use it, so it will be appropriate for sharing.

Perhaps in your little corner of the world MS documents still reign supreme. But can your MS Word open a.doc written by your Mom in 1995, allow you to add commentary, and then save it back into the archive in its original format?

Short answer: Microsoft's breakage of its own standards to leverage its marketing position has you screwed. You might not know that yet, but you are definitely screwed.

Hop off that dinosaur, its in its death throes (beware that thrashing tail). Get on some critter that has some l

You're clearly misinformed. ODF was the first document format ever to become an industry standard (ISO/IEC 26300 [iso.org]).

Microsoft then suddenly decided it also wanted to be kind of open and standardized and drafted ISO/IEC 29500 [iso.org]

There have been lots of discussions about ISO/IEC 29500 also on slashdot, because of the lack of necessity of another ISO document standard and how MS got it approved, the lack of a reference implementation (Office 2007 wasn't OOXML compliant), the reference to software patents within the standard and the way ISO 29500 got approved.

The way MS acted when getting OOXML ISO approved is just one of the reasons why I always have "Fat Tony's" voice in my head when reading their public statements.

What does "human readable" mean for you? I know that if I stare at a flash drive, I can't read the files. I know if I click on the (GUI representation of the) file to open it, some software program has to load in order for me to read the file. If the software loads quickly, why should I care whether it's a terminal program, a notepad, or Writer?

ASCII was incorporated into the Unicode character set as the first 128 symbols, so the 7-bit ASCII characters have the same numeric codes in both sets. This allows UTF-8 to be backward compatible with 7-bit ASCII, as a UTF-8 file containing only ASCII characters is identical to an ASCII file containing the same sequence of characters. Even more importantly, forward compatibility is ensured as software that recognizes only 7-bit ASCII characters as special and does not alter bytes with the highest bit set (as is often done to support 8-bit ASCII extensions such as ISO-8859-1) will preserve UTF-8 data unchanged.

Please describe all the platforms you presently use which do not support ASCII, and please provide statistics on the market presence for such platforms.

You can't assume that ASCII will be more readable than any other binary format.

Yes we can, the combination of simplicitly and ubiquity mean it is highly unlikely we will lose the ability to read it.

UTF-8 is a little more complex but the encoding method can still be described in less than a page, the harder bit is what to do with the code point sequence you get from decoding but for most widely used languages* that is a simple table lookup.

Do you have any tools that can open and read PETSCII?

Well you might end up with swapped case and block-drawing would be a mess but you could read the actual text by just treating the file as ASCII.

ASCII is more than 30 years old, it's 51 years old, and I'd bet $10,000 that it will be readable by nearly every computer in another 51 years. UTF-8 and UTF-16 are also highly unlikely to be unreadable anytime during my lifetime since they've been in use for 21 years and are open standards with many real world implementations.

ODT contains printable characters. Unzip an.odt file - all the content is XML. Of course there may also be pictures and diagrams in there too but that's why its a zip file in the first place. But perhaps you mean human only characters. Well throw the content through pandoc or any other converter.

If the software loads quickly, why should I care whether it's a terminal program, a notepad, or Writer?

If loading quickly is important, than Open/LibreOffice Writer doesnt fit the bill. Even on a very fast SSD that thing takes several seconds to load. My guess is Microsoft Word plays on the same bloated field.

Am thinking if part of it is preloaded at boot-time, it would load a lot quicker; I'm told that this happens on Windows with MS Office for the same reason, which could explain why the work-laptop I was using earlier would load MS Office fairly quickly, yet take half a minute to boot:)

I'm probably too old for your definition of quickly. I find that taking a flash drive out of my pocket and plugging it into a USB port takes several seconds. I also find that bringing up a file browser and locating the file I want to look at takes several seconds. If the amount of time it takes to show me the file contents is on par with the amount of time I take from when I decide I want to see a particular file until the time I actually click on its icon, I'm sufficiently happy with that.

Future-proofing. It means that if the digital archaeologists of the year 3000 dig up the old british archives from underneath the remains of old London, they have a decent chance of figuring out how to read it. ODF uses a zip container, but within that the actual text is in a format that can be figured our from scratch even if the spec is lost.

but within that the actual text is in a format that can be figured our from scratch even if the spec is lost.

So I'm guessing that if I compose my document using 128 character ascii, then the entire unzipped ODF file would be my document + XML, all using 128 character ascii - whereas if I compose my document using a different Unicode font (for example, if I want it to display Korean characters), then the unzipped ODF file would consist of my document in Unicode + XML in 128 character ascii. Does that sound correct to you?

ASCII is a subset of UTF-8-encoded Unicode. If you do not use anything beyond ASCII in your document, the unzipped file will only contain ASCII. If you put Korean characters into your document (and you do not have to change font to do so, if you are using a decent font), the unzipped file will contain non-ASCII characters. In both cases, the file will be a valid UTF-8-encoded Unicode XML document.

Saying a file contains Korean is a meaningless statement say unless the doc unambiguously tells you the encoding. Otherwise you're just guessing. If the XML file says it is encoded as US-ASCII but contains shift bits or extended chars then your XML parser would be fully within its rights to throw your non-compliant file out on its ass. If you're lucky it would allow the chars through but it would still be up to your app to heuristically figure out what they meant. So no you can't just shove some Korean in t

Human readable means "write it to 7 track 1/2" tape at 556 bits per inch, sprinkle iron filings on the tape and look at the bits with a magnifying glass. Or maybe its not 1972 any more and you can use you phone. Choose one.

For what it's worth, ODF is XML, which nominally human readable. So is Microsoft's OOXML, a perversion that demonstrates clearly that "human readable" doesn't always mean what it says. The main difference between ODF and OOXML is that ODF actually is a credible attempt to be open and portable whereas OOXML is designed to achieve the opposite.

OOXML wasn't designed as an exchange format at all ; every indication is that it's just an XML serialization of the internal data structures of Office. (The "Strict" version that nothing can write was produced after removing some of the more egregious kludges that have accumulated over time in Office).

The only thing it was designed to achieve was to provide some reasonable doubt that it might be an "open format", at a time when open formats were starting to become all the rage.

Because Office allegedly "supports" ODF, that reasonable doubt is sadly still there.

Not so much any more.

The real version of MS Office doesn't run on most of the computing platforms people want to use. Instead there's real competition, with dozens of variably capable Office tools available. On Android, you can get QuickOffice, Polaris Office, Kingsoft Office OfficeSuite and even the almost full version of Open Office. On iOS there's the Apple collection as well as Office HD and a some of the same Android apps. Even the web suites like Drive and Office Online work well enough.

Not that it really matters so much, the only problem I had was finding a library to write a.ods file (basically wanted to write a csv, but in a format that Excel would actually fucking render correctly, the fucker). Writing out.xls files was just not available unless I had Office installed and called some COM wrapper to some craziness.

The main reason you might want a human readable format is for collaboration ;

So many of my customers have collaborative content editing requirements as follows

* All changes to be auditable* Changes to be peer reviewed before going into the released content

Which basically screams out to be put in a version control system ; the problem is that merging sucks for binary blob formats.

You can close the gap either by creating better merge tools that understand your blobs, or moving the document structure to line-based text that merges well ; for a document of any complexity, you're going to need the improved merge tools, but line-based text makes sense for those who can read it without the GUI tools.

As programmers we fill the role of that improved merge tool for the content that we manage ; we forget that for most people, parsing and grokking even something as simple as nicely prettified HTML is akin to reading Sanskrit blindfolded from stone tablets wearing gloves.

I agree though, I want to move most of my technical authors to Markdown so that I can have an easy platform for converting their content to multiple formats for consumption.

The vast majority of their users aren't especially smart when it comes to technology. They're essentially office workers - they don't give a stuff about the underlying format, they only care about being able to do their job.

So true. And therefore we should be thankful that some knowledgeable people who do care about such important matters are willing to step forward to do the right thing.

I thought the whole point of ODF is that it was readable without certain software. All you need is to unzip it and you can look at the underlying XML files, which is a hell of a alot better than doc was, and the XML in ODF is more readable/user modifiable than OOXML (in my opinion (I've opened it and modified it myself on a few occasions)).

At least how I heard it, back when Massachusetts was going to use it that was a big part of the reason (documents still readable even if the software is long gone).

Why do we have to use something so complicated and unreadable without certain software? Something like markdown or even LaTeX if you have smart users would be better.

A bit condescending there. "Smart users" might prefer their time to be spent more productively with a WYSIWYG word processor than learning some stupid markup language just because the file format is potentially a bit simpler.

Besides, I'm sure someone could produce an ODT to Markdown / Latex tool if they wished. Both sides are fairly well documented and open standards after all.

Yes, we need a C++ ISO standard to make sure that all the compilers comply with C++11. Oh wait, Microsoft still can't figure out how to support C++11 fully. The MSDN blog cites "resource constraints" as the problem. How that fits in with laying off 18,000 employees, I'm not sure.

When the european commission does a public consultation, it get aound 300 comments (even when the policy could have worldwide implications). Most of them are from companies trying to keep some kinf of status quo, then from NGO saying it doesn't do enough and very few people answer under their own name (eg: DG CLIMA consultations [europa.eu]. In this case, 500 comments is a very good return rate...

He is a fuckwit that raises the point ; what if this is just a dastardly plan to get public orgs to pay for an upgrade to Office 2013? It could be regarded as the low-risk option - and lower versions do not support ODF 1.2.

The likelihood is that they have Software Assurance anyway so are covered for whatever version of MS Office they choose to run on whatever version of Windows they want to support.

If the IS staff deem an upgrade required there will be a time/people cost of sorts (although remember the staff will be employed doing X anyway so it's more a scheduling priority) but not an licensing upgrade cost.