NSA's Super-Secure Database Dodges Bullet From Senate

Share

NSA's Super-Secure Database Dodges Bullet From Senate

The sweeping database software that stores top-secret information inside the National Security Agency may yet be adopted by the rest of the U.S. Defense Department and other government agencies, after a change to the proposed Defense Department budget for the coming year.

This week, the leaders of the House and Senate Armed Services committees made their final changes to the National Defense Authorization Act – the Congressional bill detailing the Defense Department budget for 2013 – and this included the removal of language that threatened to curb the use Accumulo, the massive database developed by the NSA and then shared with the world as open source software.

Though Accumulo includes security controls you won't find in other databases designed to store such large amounts of data, the Senate Armed Services committee had questioned whether it ran afoul of a government policy that bars agencies from building their own software when they can just as easily use commercial alternatives. In an earlier version of the DoD budget, the Senate even went so far as to order the director of the NSA to merge Accumulo's security tools into other open source database projects.

Some feared the committee's stance would set a dangerous precedent for the treatment of open source software inside the government, but at least some of these fears have been allayed by the final version of the bill. The bill still requires approval from Congress as a whole and President Obama, but according to The Hill, Senate Armed Services Committee Chairman Carl Levin told reporters on Tuesday that he saw no reason for the White House to veto the bill as it now stands. Senator Levin's office did not immediately respond to requests for comment.

>'If you talk to people around the government, the joke is: "I don't want to build new software. I might have to face a Congressional hearing." You can see how people might start to think twice about trying to solve new problems.'

— Oren Falkowitz

The new language is certainly a big win for Sqrrl, a company founded by a group of NSA engineers who helped build Accumulo. The startup seeks to bring the NSA's database to other government agencies and businesses in much the same way a company like Red Hat delivered the Linux operating system to the commercial market. But Oren Falkowitz – one of the ex-NSA engineers that founded Sqrrl – still wonders whether the controversy around Accumulo will have a "chilling effect" on the way government agencies think about building and open sourcing new software.

"If you talk to people around the government, the joke is: 'I don't want to build new software. I might have to face a Congressional hearing,'" Falkowitz says. "You can see how people might start to think twice about trying to solve new problems."

Multiple Senate Armed Service committee staffers did not immediately responded to requests for comment, but in joint House-Senate statement detailing changes to the DoD bill, those overseeing the bill indicated that although they're not opposed to the use of open source software inside the government, they don't want agencies taking things too far.

"Recently, because of market trends and opportunities, DOD organizations are more reluctant to buy licensed commercial software products using traditional licensing models, in part due to the availability and attractiveness of open-source software," the statement reads. "This trend overall is positive in that it puts pressure on industry to make better products more economically. However, [we] believe it is also possible for government-funded, essentially in-house development programs that unjustifiably compete with the private sector to spring up under the 'open source' banner."

Based on BigTable – a massive database that underpins parts of Google's online empire – Accumulo is a means of storing vast amounts of data across thousands of ordinary computer servers. But unlike Google's database, it can provide separate security controls for each individual piece of data – something known as "cell-level security."

Basically, this means you can ensure that each person using the database can only access pieces of information they're authorized to access, and according to a speech last fall by General Keith Alexander, the director of the NSA, the agency is now using this sweeping database and its fine-grained security controls – though he did not provide particulars.

The government has a kind of love/hate relationship with this sort of open source project. Many important open source tools have emerged from government agencies, but they often face bureaucratic hurdles you don't see in the commercial world. In this case, Accumulo raised the ire of the Senate Armed Service committee, which felt the NSA had duplicated work done by existing open source projects, including HBase and Cassandra, two other efforts to reproduce Google BigTable. The committee oversees the Department of Defense, which includes the NSA.

According to a Senate Armed Services committee staffer who spoke to Wired in June under the condition that his name not be revealed because he isn't authorized to speak with the press, the committee had no intention of preventing the NSA from using Accumulo – and it realized the government manpower had already been spent in building the database. But, the staff said, it didn't want other parts of the DoD using Accumulo if there were more-active communities behind projects like Hbase and Cassandra.

In its earlier version of the DoD budget, the committee barred the rest of the DoD from using Accumulo unless they could prove that Accumulo was a “successful ... open source database with adequate industry support and diversification.”

For Gunnar Hellekson – a chief technology strategist at Red Hat who closely follows the government's approach to open source software – this language posed a threat not only to Accumulo but to open source project across the government. "It doesn’t take much imagination to see that same ‘adequacy criteria’ applied to all open source software projects,” Hellekson wrote earlier this year. "Got a favorite open source project on your DoD program, but no commercial vendor? Inadequate. Only one vendor for the package? Lacks diversity. Proprietary software doesn’t have a burden like this."

From where Hellekson was sitting, it was obvious that Accumulo was very different from the likes of Hbase and Cassandra. "When Accumulo was written, it was definitely doing new work," he told us. “Some of its differentiating features are being handled by other pieces of software. But other core concepts are unique, including the cell-level security.... That’s an incredibly important feature, and to do it properly is incredibly complicated."

But it appears the Senate has now backed down. In that joint House-Senate statement on the DoD bill, Accumulo is cited by name. "[The Department of Defense] has already determined that the Accumulo database that NSA developed using government and contract engineers is a successful open-source project that is supported by commercial companies," the statement read. "[We] expect that future acquisitions of Accumulo would be executed through such commercial vendors."

Those commercial vendors include Sqrrl. But Oren Falkowitz isn't quite ready to celebrate. "Obama still has to sign it," he says. "I wouldn't jump for joy until it's actually a law."