The black box: Inside America's massive new surveillance centre

Before yottabytes of data can begin piling up inside the servers
of the NSA's new centre, they must be collected. To achieve that
more efficiently, the agency has installed secret
electronic-monitoring rooms in major US telecom facilities. These
are where the agency taps into the US communications networks, a
practice that came to light during the Bush years but was never acknowledged by the agency. The broad
outlines of the so-called warrantless-wiretapping programme have
long been exposed -- how the NSA secretly and illegally bypassed
the Foreign Intelligence Surveillance Court, which was supposed to
oversee and authorise highly targeted domestic eavesdropping; how
the programme allowed monitoring of millions of American phone
calls and email. In the wake of the programme's exposure, Congress
passed the FISA Amendments Act of 2008, which largely made the
practices legal. Telecoms that had agreed to participate in the
illegal activity were granted immunity from prosecution and
lawsuits. What hasn't been revealed until now, however, was the
size of this domestic spying programme.

For the first time, a former NSA official has gone on the record
to describe the programme, codenamed Stellar Wind, in detail.
William Binney was a senior crypto-mathematician responsible for
automating the agency's worldwide listening network. A tall man
with dark, determined eyes behind thick-rimmed glasses, the
68-year-old spent nearly four decades breaking codes and finding
new ways to channel billions of private phone calls and email
messages from around the world into the NSA's bulging databases. As
chief and one of the two cofounders of the agency's Signals
Intelligence Automation Research Center, Binney and his team
designed much of the infrastructure that's still probably in
use.

He explains that the agency could have installed its gear at the
nation's cable landing stations -- the two dozen or so sites where
fibre-optic cables come ashore. If it had, the NSA could have
limited its eavesdropping to international communications, which at
that time was all that was allowed under US law. Instead it put
wiretapping rooms at key junctions throughout the country, thus
gaining access to most of the domestic traffic. The network of
intercept stations, or "switches", goes far beyond the room in an
AT&T building in San Francisco exposed by a whistleblower in
2006. "I think there's ten to 20 of them," Binney says. "Not just
San Francisco; they have them in the middle of the country and on
the East Coast."

Listening in doesn't stop at the telecom switches. To capture
satellite communications, the agency also monitors AT&T's
powerful earth stations, satellite receivers in locations that
include Roaring Creek and Salt Creek. Tucked away in rural
Pennsylvania, Roaring Creek's three 32-metre dishes handle much of
the country's communications to and from Europe and the Middle
East. And on a remote stretch in Arbuckle, California, three
similar dishes at the company's Salt Creek station service the
Pacific Rim and Asia.

Binney left the NSA in late 2001, shortly after the agency
launched its warrantless-wiretapping programme. "They violated the
[US] Constitution setting it up," he says. "But they didn't care.
They were going to do it, and they were going to crucify anyone who
stood in the way. When they started violating the Constitution,
I couldn't stay." Binney says Stellar Wind was larger than has
been disclosed and included listening to domestic phone calls as
well as inspecting domestic email.
At the outset the programme recorded 320 million calls a day, he
says -- about 73 to 80 per cent of the total volume of the agency's
worldwide intercepts.

The haul only grew. According to Binney -- who has kept close
contact with agency employees until a few years ago -- the taps in
the secret rooms dotting the country are powered by software
programs that conduct "deep packet inspection", examining internet
traffic as it passes through the ten-gigabit-per-second cables at
the speed of light.

The software, created by a company called Narus that's now part
of Boeing, is controlled from NSA headquarters at Fort Meade in
Maryland and searches US sources for addresses, locations,
countries and phone numbers, as well as watch-listed names,
keywords and phrases in emails. Any communication that arouses
suspicion, especially those to or from the million or so people on
agency watch lists, is recorded and transmitted to the NSA. The
scope expands from there, Binney says. Once a name is entered into
the Narus database, all communications to and from that person are
routed to the NSA's recorders. "If your number's in there? Routed
and gets recorded." And when Bluffdale is completed, whatever is
collected will be routed there.

According to Binney, one of the deepest secrets of the Stellar
Wind programme -- again, never confirmed until now -- was that the
NSA gained warrantless access to AT&T's domestic and
international billing records. As of 2007, AT&T had more than
2.8 trillion records in a database at its Florham Park, New Jersey,
complex. Verizon was also part of the programme. "That multiplies
the call rate by at least a factor of five," Binney says. "So
you're over a billion and a half calls a day." (Verizon and
AT&T said they would not comment on matters of national
security.)

After he left the NSA, Binney suggested a system for monitoring
people's communications according to how closely they are connected
to a target. The further away from the target -- say just an
acquaintance of a friend of the target -- the less the
surveillance. But the agency rejected the idea, and, given the
massive new storage facility in Utah, Binney suspects that it now
simply collects everything. He says: "They're storing everything
they gather."

Once communications are stored, the datamining begins. "You can
watch everybody all the time with datamining," Binney says.
Everything a person does is charted on a graph, "financial
transactions or travel or anything", he says. Thus the NSA is able
to paint a detailed picture of someone's life. The NSA can also eavesdrop on phone calls directly and in real time. According
to Adrienne Kinne, who worked before and after 9/11 as a voice
interceptor at the NSA facility in Georgia, in the wake of the
World Trade Center attacks "basically all rules were thrown out the
window, and they would use any excuse to justify a waiver to spy on
Americans". Even journalists calling home from overseas were
included. "A lot of time you could tell they were calling their
families," she says. "Intimate, personal conversations." Kinne
found eavesdropping on innocent citizens distressing. "It's like
finding somebody's diary," she says.

But there is reason for everyone to be distressed about the
practice. Once the door is open for the government to spy on US
citizens, there are temptations to abuse that power for political
purposes, as when Richard Nixon eavesdropped on his political
enemies during Watergate and ordered the NSA to spy on anti-war
protesters. Those and other abuses prompted Congress to enact
prohibitions in the mid-1970s against domestic spying.

Before he left the NSA, Binney tried to persuade officials to
create a more targeted system that could be authorised by a court.
At the time, the agency had 72 hours to obtain a legal warrant;
Binney devised a method to computerise the system. But such a
system would have required close co-ordination with the courts, and
NSA officials weren't interested, Binney says. Asked how many
communications -- "transactions", in NSA's lingo -- the agency has
intercepted since 9/11, Binney estimates "between 15 and 20
trillion over 11 years".

Binney hoped that Barack Obama's new administration might be
open to addressing constitutional concerns. He and another former
senior NSA analyst, J Kirk Wiebe, tried to explain an automated
warrant-approval system to the Department of Justice's inspector
general. They were given the brush-off. "They said, oh, OK, we
can't comment," Binney says. Sitting in a restaurant not far from
NSA headquarters, the place where he spent nearly 40 years of his
life, Binney held his thumb and forefinger close together. "We are,
like, that far from a turnkey totalitarian state," he says.

US Army Corps of engineers conceptual site plan / Wired

There is still one technology preventing untrammelled government
access to private digital data: strong encryption. Anyone -- from
terrorists and weapons dealers to corporations, financial
institutions and ordinary email senders -- can use it to seal their
messages, plans, photos and documents in hardened data shells. For
years, one of the hardest shells has been the Advanced Encryption
Standard (AES), one of several algorithms used by much of the world
to encrypt data. Available in three different strengths -- 128
bits, 192 bits and 256 bits -- it's incorporated in most commercial
email programs and web browsers and is considered so strong that
the NSA has even approved its use for top-secret US government
communications. Most experts say that a so-called brute-force
computer attack on the algorithm -- trying one combination after
another to unlock the encryption -- would likely take longer than
the age of the universe. For a 128-bit cipher, the number of
trial-and-error attempts would be 340 undecillion (1036).

Breaking into those complex mathematical shells like the AES is
one of the key reasons for the construction going on in Bluffdale.
That kind of cryptanalysis requires two major ingredients: super-fast
computers to conduct brute-force attacks on encrypted messages
and a massive number of those messages for the computers to
analyse. The more messages from a given target, the more likely it
is for the computers to detect telltale patterns, and Bluffdale
will be able to hold a great many messages. "We questioned it one
time," says another source, a senior intelligence manager who was
also involved with the planning. "Why were we building this NSA
facility? And, boy, they rolled out all the old guys -- the crypto
guys." According to the official, these experts told then-director
of national intelligence Dennis Blair, "You've got to build this
thing because we just don't have the capability of doing the
code-breaking." It was a candid admission. In the long war between
the code breakers and the code makers -- the tens of thousands of
cryptographers in the worldwide computer- security industry -- the
code breakers were admitting defeat.

So the agency had one major ingredient -- a massive data-storage
facility -- under way. Meanwhile, across the country in Tennessee,
the US government was working in utmost secrecy on the other vital
element: the most powerful computer the world has ever known.The
plan was launched in 2004 as a modern-day Manhattan Project. Dubbed
the High Productivity Computing Systems programme, its goal was to
advance computer speed a thousandfold, creating a machine that
could execute a quadrillion (1015) operations a second, known as a
petaflop -- the computer equivalent of breaking the land speed
record. And as with the Manhattan Project, the venue chosen for the
supercomputing programme was the town of Oak Ridge in eastern
Tennessee, a rural area where sharp ridges give way to low,
scattered hills, and the southwestward-flowing Clinch River bends
sharply to the southeast. About 40km from Knoxville, it is the
"secret city" where uranium-235 was extracted for the first atomic
bomb. A sign near the exit said: "What you see here, what you do
here, what you hear here, when you leave here, let it stay here."
Today, not far from where that sign stood, Oak Ridge is home to the
Department of Energy's Oak Ridge National Laboratory, and it's
engaged in a new secret war.

Such an advance would open a window on a trove of
foreign secrets

In 2004, as part of the supercomputing programme, the Department
of Energy established its Oak Ridge Leadership Computing Facility
for multiple agencies to join forces on the project. But in reality
there would be two tracks, one unclassified, in which all of the
scientific work would be public, and another top secret, in which the NSA could pursue its own computer
covertly. "For our purposes, they had to create a separate
facility," says a former senior NSA computer expert who worked on
the project and is still associated with the agency. (He is one of
three sources who described the programme.) It was an expensive
undertaking.

Known as the Multiprogram Research Facility, or Building 5300,
the $41 million, five-storey, 20,000m2 structure was built on a
plot of land on the lab's East Campus and completed in 2006.
Inside, 318 scientists, computer engineers and other staff work in
secret on the cryptanalytic applications of high-speed computing
and other classified projects. The centre was named in honour of
George R Cotter, the NSA's now-retired chief scientist and head of
its information technology programme. Not that you'd know it.
"There's no sign on the door," says the ex-NSA computer expert.

At the DOE's unclassified centre at Oak Ridge the team had its
Cray XT4 supercomputer upgraded to a warehouse-sized XT5. Named
Jaguar for its speed, it clocked in at 1.75 petaflops and was the
world's fastest computer in 2009.

Meanwhile, over in Building 5300, the NSA succeeded in building
an even faster supercomputer. "They made a big breakthrough," says
another former senior intelligence official, who helped oversee the
programme.

The NSA's machine was probably similar to the unclassified
Jaguar, but it was much faster out of the gate, modified
specifically for cryptanalysis and targeted against one or more
specific algorithms, like the AES. They were moving from the R&D
phase to actually attacking extremely difficult encryption
systems.

The codebreaking effort was up and running.

The agency pulled the shade down on the project, says the former
official. "Only the chairman, vice chairman and the two staff
directors of each intelligence committee were told," he says. "They
were thinking this was going to give them the ability to crack
current public encryption."

In addition to giving the NSA access to a tremendous amount of
Americans' personal data, such an advance would also open a window
on a trove of foreign secrets. Whereas today most sensitive
communications use the strongest encryption, much of the older data
stored by the NSA, including a great deal of what will be
transferred to Bluffdale once the centre is complete, is encrypted
with more vulnerable ciphers. "Remember," says the former
intelligence official, "a lot of foreign government stuff we've
never been able to break is 128[-bit] or less. Break all that and
you'll find out a lot more of what you didn't know -- stuff we've
already stored -- so there's an enormous amount of information
still in there."

That, he notes, is where the value of Bluffdale and its
mountains of long-stored data will come in. What can't be broken
today may be broken tomorrow. "Then you can see what they were
saying in the past," he says. "By extrapolating the way they did
business, it gives us an indication of how they may do things now."
The danger, the former official says, is that it's not only foreign
government information that is locked in weaker algorithms; it's
also a great deal of personal domestic communications, such as
Americans' email intercepted by the NSA in the past decade.

But first the supercomputer must break the encryption, and to do
that, speed is everything. The faster the computer, the faster it
can break codes. The Data Encryption Standard, the 56-bit
predecessor to the AES, debuted in 1976 and lasted about 25 years.
The AES made its first appearance in 2001 and is expected to remain
strong and durable for at least a decade. But if the NSA has
secretly built a computer that is considerably faster than machines
in the unclassified arena, then the agency has a chance of breaking
the AES in a much shorter time. And with Bluffdale in operation,
the NSA will have the luxury of storing an ever-expanding archive
of intercepts until that breakthrough comes along.

But despite its progress, the agency has not finished building
at Oak Ridge, nor is it satisfied with breaking the petaflop
barrier. Its next goal is to reach exaflop speed, one quintillion
(1018) operations a second, and eventually zettaflop (1021) and
yottaflop.

These goals have considerable support in Congress. Last November
a bipartisan group of 24 senators sent a letter to President Obama
urging him to approve continued funding through 2013 for the
Department of Energy's exascale computing initiative (the NSA's
budget requests are classified). They cited the necessity to keep
up with and surpass China and Japan. "The race is on to develop
exascale computing capabilities," the senators noted. By late 2011
the Jaguar (now with a peak speed of 2.33 petaflops) ranked third
behind Japan's "K Computer", with 10.51 petaflops, and the Chinese
Tianhe-1A system, with 2.57 petaflops.

But the real competition will take place in the classified
realm. To secretly develop the new exaflop (or higher) machine by
2018, the NSA has proposed constructing two connecting buildings,
totalling 24,100m2, near its current facility on the East Campus of
Oak Ridge. Called the Multiprogram Computational Data Center, the
buildings will be low and wide like giant warehouses, a design
necessary for the dozens of computer cabinets that will compose an
exaflop-scale machine, possibly arranged in a cluster to minimise
the distance between circuits. According to a presentation
delivered to DoE employees in 2009, it will be an "unassuming
facility with limited view from roads", in keeping with the NSA
secrecy. And it will have an extraordinary appetite for
electricity, using about 200 megawatts, enough to power 200,000
homes. In the meantime Cray is working on the next step for the
NSA, funded in part by a $250 million contract with the Defense
Advanced Research Projects Agency. It's a massively parallel
supercomputer called Cascade, a prototype of which is due at the
end of 2012. Its development will run alongside the unclassified
effort for the DoE and other partner agencies. That project, due in
2013, will upgrade the JaguarXT5 into an XK6, codenamed Titan,
upping its speed to ten to 20 petaflops.

Yottabytes and exaflops, septillions and undecillions -- the
race for computing speed and data storage goes on. In his 1941
story The Library of Babel, Jorge Luis Borges imagined a collection
of information where the entire world's knowledge is stored but
barely a single word is understood. In Bluffdale the NSA is
constructing a library on a scale that even Borges might not have
contemplated. And to hear the masters of the agency tell it, it's
only a matter of time until every word is illuminated.

Comments

No doubt, David. Excellent point. I might as well toss my Time Machine and the $200+ into the garbage.

evelyn

May 20th 2012

Lets screw with the Governments collective head. Everyone should encrypt everything. I was thinking of writing an app that would also encrypt the phone conversations. We are making it way to easy for them. Make them work for the information they steal.

Bill

Mar 30th 2012

A yottabyte of information, how will they find the bit them want to read?

John

Mar 30th 2012

This is way beyond stupid.the sheer amount of data collected is self defeating regardless of software to analyze it.

So what if you know all my details what I ate this morning.

A simple way to defeat the law breaking criminals in the NSA is to encrypt everything with AES 256 and then encrypt it again with AES 256.

All your efforts in vain.

Ya boo sucks to you.

tjoker

Apr 1st 2012

Let's all go proxy now!!BTW: "taken from the May 2012 issue of Wired magazine"? That's still a month ahead!

Maszek

Apr 2nd 2012

James,

Thanks for keeping an eye on big data :)

GetCocoon

Apr 2nd 2012

Why are all the above comments negative? I think its about time, but it is none of our business where the facility is located. Another Area 51, looks like Area 52.

not needed

Apr 3rd 2012

There are so many areas such as these globally.

Wlazsek

Apr 6th 2012

Start writing letters unless you have already been "tagged"!

Countermeasures1

Apr 8th 2012

and open email and facebook accounts in false names

Johanthon Smith

Apr 8th 2012

In reply to Johanthon Smith

such as, oh idk, Johanthon Smith? xD

billyjones

Sep 4th 2012

All this technology can be defeated by a post it note delivered in the hand of wino courier with a $20 bill as payment in his pocket.

Holland

Aug 1st 2012

All this technology can be defeated by a post it note delivered in the hand of wino courier with a $20 bill as payment in his pocket.

Holland

Aug 1st 2012

Idiots are collecting more hay making it harder to find the needleI will look into full encryption options