Share this story

President Barack Obama issued an executive order today that aims to make "open and machine-readable" data formats a requirement for all new government IT systems. The order would also apply to existing systems that are being modernized or upgraded. If implemented, the mandate would bring new life to efforts started by the Obama administration with the launch of Data.gov four years ago. It would also expand an order issued in 2012 to open up government systems with public interfaces for commercial app developers.

"The default state of new and modernized Government information resources shall be open and machine readable," the president's order reads. "Government information shall be managed as an asset throughout its life cycle to promote interoperability and openness, and, wherever possible and legally permissible, to ensure that data are released to the public in ways that make the data easy to find, accessible, and usable." The order, however, also requires that this new "default state" protect personally identifiable information and other sensitive data on individual citizens, as well as classified information.

Broadening the “open” mandate

The president's mandate was initially pushed forward by former Chief Information Officer of the United States Vivek Kundra. In May of 2009, Data.gov launched with an order that required agencies to provide at least three "high-value data sets" through the portal.

However, the data sets initially published through Data.gov were in a vast assortment of formats and were entirely static "dumps." There was a great deal of resistance from agencies initially, in part because the new requirements came without any funding to produce the open sets from systems that were largely built on a patchwork of legacy systems and custom-formatted data. In August of 2010, the Obama administration created the position of "Data.gov evangelist" to push forward publishing efforts; the position was given to Jeanne Holm, former chief knowledge architect at NASA's Jet Propulsion Laboratory.

After the departure of Vivek Kundra, new federal CIO Steven VanRoekel took point on open data, pushing to expose live government data rather than static data sets. Last May, the White House ordered agencies to create public APIs that could be used by government and private developers to tap in to data and make specific “applicable Government information open and machine-readable by default.” At that time, VanRoekel said in an Office of Management and Budget blog post, "To make sure there’s no wrong door for accessing government data, we will transform Data.gov into a data and API catalog that in real time pulls directly from agency websites." This February, the Obama administration pushed open requirements forward on the scientific front and moved to make more federally funded research data available to the public to preserve "digitally formatted scientific data" for public use.

But these orders, and the efforts by federal IT leaders, have thus far not opened up much of the core of federal data to daylight. With the new executive order issued today, the Obama administration is pushing the goal posts back further for agencies in the hopes of greater IT efficiency and transparency—and of creating an ecosystem of businesses that turn mashups of government data into a profitable business.

Three months to launch

The order calls for the Office of Management and Budget to issue an Open Data Policy and sets a three-month timeline for it to be incorporated into the performance goals for all the government agencies affected by the order. Independent agencies, such as the Federal Communications Commission, are "requested to adhere" to the order.

Within 30 days of the publication of that policy, the order requires VanRoekel's office and Federal Chief Technology Officer Todd Park to publish "an open online repository of tools and best practices" to help agencies integrate open data standards into their systems. Within 90 days of the issuance of the policy, it will be integrated into the Office of Management and Budget's rules governing the way agencies purchase IT systems and services.

Just how effective this order will be in the face of the government's ongoing budget crisis is unclear. With sequestration cutting back many programs, there's little maneuvering room for agencies to make significant changes to the systems that this order would affect.

Share this story

Sean Gallagher
Sean is Ars Technica's IT and National Security Editor. A former Navy officer, systems administrator, and network systems integrator with 20 years of IT journalism experience, he lives and works in Baltimore, Maryland. Emailsean.gallagher@arstechnica.com//Twitter@thepacketrat

Major government data in economics and biological science is already available on a large scale. But every kind of data has its own idiosyncratic format. Each one can be made machine readable with a modest programmer effort. But I doubt there is any realistic hope of some kind of standard set of formats.

I'm a pretty consistent liberal, but the president's policies have really worried me lately (Gitmo, drone strikes, signing the PATRIOT Act, etc.), this is definitely a good move. My biggest concern is competence in creating these APIs and ensuring that the data is only exposed to the correct people.

And all federally funded research should require the publication of complete datasets in machine-readable form. Any statistic or numeric output mentioned in the paper must be accompanied by all of the data necessary to derive it. That way anyone can find mistakes and/or reproduce the findings.

*If* this is a sincere move by Obama, it could be one of the better changes he has implemented in his time in office. The interesting thing is that, even though I'm usually extremely cynical about Obama (even as a raging progressive), I cannot find any ulterior motives this time... especially because of the ongoing sequestration and how mad everyone is at him anyway. I hope it is legit and that this will eventually lead to more transparency.

The good thing about executive orders (which bypass Congress) is that without funding they have all of the teeth of a Congressional resolution. I especially like the "x shall be done..." wording in the orders, which reminds me of "Let there be light" complexes or Julius-Caesar type decrees and proclamations. It's too bad we can't hear a brace of trumpets blasting at the same time these words are read from on high... But...

Quote:

The order, however, also requires that this new "default state" protect personally identifiable information and other sensitive data on individual citizens, as well as classified information.

...this is going to be the real problem, imo. This is going to create a hacker's paradise in which nobody's info will remain confidential for long. Even if the order says, "Illegal access to personal information in these archives is forbidden," I don't think that will make a lot of difference. When has it ever? Obama's stuff is often a double-edged sword, it seems.

I hope they open the FCC ULS database (including geographic overlay). Then I could start updating my spectrum mapper again. I had to quit in January after the FCC stopped updating the FCC Spectrum Dashboard app.

In August of 2010, the Obama administration created the position of "Data.gov evangelist" to push forward publishing efforts; the position was given to Jeanne Holm, former chief knowledge architect at NASA's Jet Propulsion Laboratory.

You know, that's kind of a dream of mine: to go from one job with a ridiculous meaningless corporatese title to another with an even more absurd meaningless corporatese title.

The order, however, also requires that this new "default state" protect personally identifiable information and other sensitive data on individual citizens, as well as classified information.

...this is going to be the real problem, imo. This is going to create a hacker's paradise in which nobody's info will remain confidential for long.

Only if everything is accessible via an API which doesn't carry identity assertions. So not only do they have to build this as-yet-hypothetical API, they have to do it against the advice, and indeed against the written policy, of every single agency involved.

Your "hacker's paradise" assumes everyone acting against their own best interests, simultaneously.

Can I assume you're a big fan of the current system where sensitive-but-unclassified information that should be shared, isn't, because it physically can't be?

Given that a good portion of the government still uses ColdFusion (as noted in another Ars Article), and the bulk of government agencies have serious issues attracting and retaining technically skilled folks, I have my doubts about whether they will be able to manage this, or if they do, I have doubts as to the quality of implementation.

Given that a good portion of the government still uses ColdFusion (as noted in another Ars Article), and the bulk of government agencies have serious issues attracting and retaining technically skilled folks, I have my doubts about whether they will be able to manage this, or if they do, I have doubts as to the quality of implementation.

"Government agencies" don't do any of the work. Contractors do. I work in this space IRL, and let me tell you there is absolutely no shortage of very talented coders and architects working for government contractors.

And when you say "a good portion of the government uses ColdFusion" --- what "portion" are you talking about? There's a vast mix of technologies and capabilities across federal, state, local...

*If* this is a sincere move by Obama, it could be one of the better changes he has implemented in his time in office. The interesting thing is that, even though I'm usually extremely cynical about Obama (even as a raging progressive), I cannot find any ulterior motives this time... especially because of the ongoing sequestration and how mad everyone is at him anyway. I hope it is legit and that this will eventually lead to more transparency.

I think most of the problems with the current Administration are not Obama's fault. Unfortunately, as President, he is ultimately responsible.

Given that a good portion of the government still uses ColdFusion (as noted in another Ars Article), and the bulk of government agencies have serious issues attracting and retaining technically skilled folks, I have my doubts about whether they will be able to manage this, or if they do, I have doubts as to the quality of implementation.

I work in this space; choice of Adobe ColdFusion as the platform has very little to do with whether this is feasible.

In August of 2010, the Obama administration created the position of "Data.gov evangelist" to push forward publishing efforts; the position was given to Jeanne Holm, former chief knowledge architect at NASA's Jet Propulsion Laboratory.

You know, that's kind of a dream of mine: to go from one job with a ridiculous meaningless corporatese title to another with an even more absurd meaningless corporatese title.

Hey, someone has to mastermind the torrent of useless task forces and endless meetings...

Given that a good portion of the government still uses ColdFusion (as noted in another Ars Article), and the bulk of government agencies have serious issues attracting and retaining technically skilled folks, I have my doubts about whether they will be able to manage this, or if they do, I have doubts as to the quality of implementation.

Isn't that the point? An order from the top that this is a priority might put some pressure on the trailing agencies to improve their IT approach.

Given that a good portion of the government still uses ColdFusion (as noted in another Ars Article), and the bulk of government agencies have serious issues attracting and retaining technically skilled folks, I have my doubts about whether they will be able to manage this, or if they do, I have doubts as to the quality of implementation.

I work in this space; choice of Adobe ColdFusion as the platform has very little to do with whether this is feasible.

The platform isn't the issue. It's how the underlying data is exposed—databases, publications, and "live" data from various sources. There are vast information sources that we currently get access to in only limited forms. A lot of it is deep-link content you can't find with search engines. This order mandates that anything that is either built new or upgraded and modernized in any way provide for the data to be on tap for public access; theoretically, that improves the government's own ability to use the data as well, because it's cheaper for apps to be developed when there's an existing open data interchange format.

Ah, so this is the next Guantanamo (a big, shiny government mandate that gets more and more wishy-washy as the deadline approaches, until it's missed entirely, and what was formerly a "mandate" has suddenly been a "suggestion" all along)?

I like Obama, but it's getting frustrating to see all these good ideas watered down or outright quashed over and over again by corruption, borderline-intentional red tape, or a combination of both.

I love the idea of open data but it's just not there yet and this isn't going to change it =/. Unfortuntaly as someone who worked on data.gov I left that project with a bad taste in my mouth. General politics, attention to getting crap like facebook on there (before you even have a working model, not that I'm opposed to sharing the data when it works), and contractors who care more about billing hours, keeping source code away from the public, and how they look; rather than giving real answers and having the hard discussions.

Yes I'm quite biased but I can't tell you how frustrating it is to fight for up to date source code, only to be stonewalled.https://github.com/opengovplatform/ogpl-d78 months since their last commit (and that commit was not up todate) unless they forked another repo.

I love the idea of open data but it's just not there yet and this isn't going to change it =/. Unfortuntaly as someone who worked on data.gov I left that project with a bad taste in my mouth. General politics, attention to getting crap like facebook on there (before you even have a working model, not that I'm opposed to sharing the data when it works), and contractors who care more about billing hours, keeping source code away from the public, and how they look; rather than giving real answers and having the hard discussions.

Yes I'm quite biased but I can't tell you how frustrating it is to fight for up to date source code, only to be stonewalled.https://github.com/opengovplatform/ogpl-d78 months since their last commit (and that commit was not up todate) unless they forked another repo.

Hypocrite announcement, right after the successful construction plans of the NSA cyber HQ on Utah. I don't buy any globalist elite, nobody, making Internet 'open' or more 'secure'. Down rank me, don't care.

Obama is so open with other people's information and money. (Now, granted, the more open government is, the better. I'd like a thorough review of our President's resume and qualifications, because what I've seen really sucks or is sealed.)

Obama is so open with other people's information and money. (Now, granted, the more open government is, the better. I'd like a thorough review of our President's resume and qualifications, because what I've seen really sucks or is sealed.)

All he needs to qualify is be 35 or older and a natural born citizen. Please tell me you're not one of those 'birther' nuts...

Have you asked for the resume and qualifications of any other President?

Given that a good portion of the government still uses ColdFusion (as noted in another Ars Article), and the bulk of government agencies have serious issues attracting and retaining technically skilled folks, I have my doubts about whether they will be able to manage this, or if they do, I have doubts as to the quality of implementation.

Part of the problem is that most government jobs require you to either be a former federal government employee or a veteran with 3 or more years experience. Average "Joe on the Street" cant get in.

So anything the left finds inconvenient, is automatically labeled a nut? Can we open minds here and admit that the topic isn't cleared up yet. By no means did we see any clear evidence or straight up most solution to his 'birth' certificate here Mr. carldjennings.

So anything the left finds inconvenient, is automatically labeled a nut? Can we open minds here and admit that the topic isn't cleared up yet. By no means did we see any clear evidence or straight up most solution to his 'birth' certificate here Mr. carldjennings.

And again, did you ask for the birth certificate of any other President, or is it just because he's black?