Recovery.gov Double Fault: Broken Data Feeds

The relaunched recovery.gov site no longer supports automated data feeds. These feeds had allowed users of the 1.0 site to perform valued-added analyses, "the whole point of accountability and transparency" according one site user, who added, "from a software architecture standpoint, they seem to have missed a key principle here: backward compatibility."

The relaunched recovery.gov government-transparency site no longer supports automated data feeds. These feeds had allowed users of the 1.0 site to perform their own valued-added analyses, "the whole point of accountability and transparency," according one site user, an executive with a large, government systems integrator. According to that user, who asked not to be named, referring to the Recovery Accountability and Transparency Board (RATB) and lead contractor Smartronix, "from a software architecture standpoint, they seem to have missed a key principle here: backward compatibility."

RATB spokesperson Edward Pound confirmed that the relaunched site no longer offers the feeds. Pound did not know if notice had been provided to users, on-site or through another mechanism, of the discontinuation of the data-feed interface. He stressed that the Recovery Board is working hard to meet emerging user needs and improve site capabilities in furtherance of its non-political mission of promoting open government.Termination of Atom Feed

"The old version of recovery.gov had an Atom feed at the URL http://www.recovery.gov/?q=content/latest-financial-and-activity-reports.

"This Atom feed was pretty good. It provided an enumeration of stimulus obligations and disbursements by agency, program, and state, all in a single data feed. I had written an application that would pull this data down on a regular basis for purposes of performing additional valued-added analysis -- which, I believe, is the whole point of accountability and transparency.

"Since the site update of a few days ago, this Atom feed is no longer available. If you follow the aforementioned URL, it redirects to the site home page. Of course, this 'broke' the app I wrote. So I went ahead and looked at the new site to see if it moved to a new URL. The problem is, I can't seem to find it at all. It looks like they missed reimplementing this particular feed in the site redesign. The only download options I could find for financial/activity reports is MS-Excel spreadsheets (vs. an open standard like XML) -- one per agency (vs. a consolidated feed).

"I have attempted to contact the Recovery folks via their Facebook page in the event it might already exist but I missed it when I looked at the new site... but haven't heard back from anyone.

"From a software architecture standpoint, they seem to have missed a key principle here: backward compatibility. Once a system owner puts out an interface and client applications become dependent on that interface, it is very important to continue to support that interface going forward (or at a minimum to deprecate it with lots of warning and lots of lead time). It also seems odd that they would go backward in functionality from version 1.0 to 2.0, at least in the realm of data feeds."

RATB spokesperson Edward Pound said that the Recovery Board intends to offer recovery.gov data downloads as soon as is feasible. Site development contractor Smartronix did not respond to a request for comment on the broken data feeds or on my September 30 blog article reporting accessibility issues.The relaunched recovery.gov site no longer supports automated data feeds. These feeds had allowed users of the 1.0 site to perform valued-added analyses, "the whole point of accountability and transparency" according one site user, who added, "from a software architecture standpoint, they seem to have missed a key principle here: backward compatibility."

ITís tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.