… In 2004, as part of the supercomputing programme, the Department of Energy established its Oak Ridge Leadership Computing Facility for multiple agencies to join forces on the project. But in reality there would be two tracks, one unclassified, in which all of the scientific work would be public, and another top secret, in which the NSA could pursue its own computer covertly. “For our purposes, they had to create a separate facility,” says a former senior NSA computer expert who worked on the project and is still associated with the agency. (He is one of three sources who described the programme.) It was an expensive undertaking.

Known as the Multiprogram Research Facility, or Building 5300, the $41 million, five-storey, 20,000m2 structure was built on a plot of land on the lab’s East Campus and completed in 2006. Inside, 318 scientists, computer engineers and other staff work in secret on the cryptanalytic applications of high-speed computing and other classified projects. The centre was named in honour of George R Cotter, the NSA’s now-retired chief scientist and head of its information technology programme. Not that you’d know it. “There’s no sign on the door,” says the ex-NSA computer expert.

At the DOE’s unclassified centre at Oak Ridge the team had its Cray XT4 supercomputer upgraded to a warehouse-sized XT5. Named Jaguar for its speed, it clocked in at 1.75 petaflops and was the world’s fastest computer in 2009.

Meanwhile, over in Building 5300, the NSA succeeded in building an even faster supercomputer. “They made a big breakthrough,” says another former senior intelligence official, who helped oversee the programme.

The NSA’s machine was probably similar to the unclassified Jaguar, but it was much faster out of the gate, modified specifically for cryptanalysis and targeted against one or more specific algorithms, like the AES. They were moving from the R&D phase to actually attacking extremely difficult encryption systems.

The codebreaking effort was up and running

The agency pulled the shade down on the project, says the former official. “Only the chairman, vice chairman and the two staff directors of each intelligence committee were told,” he says. “They were thinking this was going to give them the ability to crack current public encryption.”

In addition to giving the NSA access to a tremendous amount of Americans’ personal data, such an advance would also open a window on a trove of foreign secrets. Whereas today most sensitive communications use the strongest encryption, much of the older data stored by the NSA, including a great deal of what will be transferred to Bluffdale once the centre is complete, is encrypted with more vulnerable ciphers. “Remember,” says the former intelligence official, “a lot of foreign government stuff we’ve never been able to break is 128[-bit] or less. Break all that and you’ll find out a lot more of what you didn’t know — stuff we’ve already stored — so there’s an enormous amount of information still in there.”

That, he notes, is where the value of Bluffdale and its mountains of long-stored data will come in. What can’t be broken today may be broken tomorrow. “Then you can see what they were saying in the past,” he says. “By extrapolating the way they did business, it gives us an indication of how they may do things now.” The danger, the former official says, is that it’s not only foreign government information that is locked in weaker algorithms; it’s also a great deal of personal domestic communications, such as Americans’ email intercepted by the NSA in the past decade.

But first the supercomputer must break the encryption, and to do that, speed is everything. The faster the computer, the faster it can break codes. The Data Encryption Standard, the 56-bit predecessor to the AES, debuted in 1976 and lasted about 25 years. The AES made its first appearance in 2001 and is expected to remain strong and durable for at least a decade. But if the NSA has secretly built a computer that is considerably faster than machines in the unclassified arena, then the agency has a chance of breaking the AES in a much shorter time. And with Bluffdale in operation, the NSA will have the luxury of storing an ever-expanding archive of intercepts until that breakthrough comes along.

But despite its progress, the agency has not finished building at Oak Ridge, nor is it satisfied with breaking the petaflop barrier. Its next goal is to reach exaflop speed, one quintillion (1018) operations a second, and eventually zettaflop (1021) and yottaflop.

These goals have considerable support in Congress. Last November a bipartisan group of 24 senators sent a letter to President Obama urging him to approve continued funding through 2013 for the Department of Energy’s exascale computing initiative (the NSA’s budget requests are classified). They cited the necessity to keep up with and surpass China and Japan. “The race is on to develop exascale computing capabilities,” the senators noted. By late 2011 the Jaguar (now with a peak speed of 2.33 petaflops) ranked third behind Japan’s “K Computer”, with 10.51 petaflops, and the Chinese Tianhe-1A system, with 2.57 petaflops.

But the real competition will take place in the classified realm. To secretly develop the new exaflop (or higher) machine by 2018, the NSA has proposed constructing two connecting buildings, totalling 24,100m2, near its current facility on the East Campus of Oak Ridge. Called the Multiprogram Computational Data Center, the buildings will be low and wide like giant warehouses, a design necessary for the dozens of computer cabinets that will compose an exaflop-scale machine, possibly arranged in a cluster to minimise the distance between circuits. According to a presentation delivered to DoE employees in 2009, it will be an “unassuming facility with limited view from roads”, in keeping with the NSA secrecy. And it will have an extraordinary appetite for electricity, using about 200 megawatts, enough to power 200,000 homes. In the meantime Cray is working on the next step for the NSA, funded in part by a $250 million contract with the Defense Advanced Research Projects Agency. It’s a massively parallel supercomputer called Cascade, a prototype of which is due at the end of 2012. Its development will run alongside the unclassified effort for the DoE and other partner agencies. That project, due in 2013, will upgrade the JaguarXT5 into an XK6, codenamed Titan, upping its speed to ten to 20 petaflops … (full long text).