HomeNewsXKeyscore: NSA tool collects 'nearly everything a user does on the Internet'

XKeyscore: NSA tool collects ‘nearly everything a user does on the Internet’

NSA analysts require no prior authorization for searches of emails, social media activity, and browsing history

August 1, 2013

A top secret National Security Agency program allows analysts to search with no prior authorization through vast databases containing emails, online chats and the browsing histories of millions of individuals — its “widest-reaching” system for developing intelligence from the Internet — according to documents provided by whistleblower Edward Snowden, Guardian columnist Glenn Greenwald reported Wednesday.

Training materials for XKeyscore detail how analysts can use it and other systems to mine enormous agency databases by filling in a simple on-screen form giving only a broad justification for the search, including U.S. persons.. The request is not reviewed by a court or any NSA personnel before it is processed.

The purpose of XKeyscore is to allow analysts to search the metadata as well as the content of emails and other internet activity, such as browser history, even when there is no known email account (a “selector” in NSA parlance) associated with the individual being targeted.

Analysts can also search by name, telephone number, IP address, keywords, the language in which the internet activity was conducted or the type of browser used. An NSA tool called DNI Presenter, used to read the content of stored emails, also enables an analyst using XKeyscore to read the content of Facebook chats or private messages.

The XKeyscore program also allows an analyst to learn the IP addresses of every person who visits any website the analyst specifies. As one slide indicates, the ability to search HTTP activity by keyword permits the analyst access to what the NSA calls “nearly everything a typical user does on the internet”.

William Binney, a former NSA mathematician, said last year that the agency had “assembled on the order of 20 trillion transactions about U.S. citizens with other U.S. citizens,” an estimate, he said, that “only was involving phone calls and emails.” A 2010 Washington Post article reported that “every day, collection systems at the [NSA] intercept and store 1.7 billion emails, phone calls and other type of communications.”

One document explains: “At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours.” In 2012, there were at least 41 billion total records collected and stored in XKeyscore for a single 30-day period.

Comments (38)

It i s TIME that we realize that the 2nd amendment was put in for a reason. Namely IF the Gov has gone corrupt. It is already way beyond that. The American people has flunked two IQ tests, by pitting av muslim, a sworn enemy of the USA in the white house. It’s time for a change, and to use his own slogan: YES WE CAN. He is a threat to all the people of this present world and to all humanity. He is screwing the US by the notes and having a laugh when doing it. Do we need a president who bows to the Saudis and to their god allah who is no god but a big lie? A god who’s prophet is a child molester and a sex criminal? It is time for a change. A time to get rid of them who worship blood, death and crime, while they cry allah is greater. Who smile at you and talks of peace while they plot for your extermination. You are being sold out for oil, and your so called president works intensively to hand over the USA in their hands.

And the NSA just lied to Congress that they only collected metadata. That’s not metadata. Also, the gigantic complex being built in Utah is way too big, even for global metadata, which could fit in a space the size of an average house. (Although the cooling units would be bigger than that ;’)

Reply to @Bigbro: Exceptional comment. Thanks. Still, at the normative level, considering the First Amendment, the Constitution, and free life in general, the USA is one of the most free countries in the world. Having said this, all of the normative aspects of freedom are at at the highest test at this point. It is important to defend the normative constitutional and Bill of Rights basis of the USA. This is why Snowden’s case was so mind capturing.

I have a question: How can we identify a “snitch”? Can one identify data-collectors via the IP addresses? Could you isolate the IP address if you suspect the “men in black”? I wonder often about a few IP addresses on my blog coming from Redmond, WA; Sunnyvale, CA, Ashburn, VA, and San Jose, CA–all simultaneously appear as soon as there is any activity on the blog. I checked the IP addresses, all of theml lead to some strange data-collecting places. I do have a very good protection system and the official web-designer that eliminated twice “hacking” activities, though, how can one recognize a “snitch”? Or remove their IP presence?

So where is your proof. I really don’t care if I am surveilled, but if I am it is not some master plan Obama brought to office. He is not a fascist. He is not droning American citizens, at least not the way you make it sound. As for whistle blowers Edward Snowden does not qualify. He was a contract employee with an agenda. He did not reveal anything any intelligent person did not already know.

this is an important topic and needs a vast amount of attention…but really what do you have to hide ? you really think you are that important that what you do will grab the govt attention ? with increasing population numbers as well as the internet i think these measures are needed. but that’s not to say there’s no need to watch these types of surveillance like a hawk to make sure our govt is for the absolute most doing it to protect it’s country and not to control

Big data and ability to correlate is not about crime, but about the influence of money. Nearly all government departments are corrupted by revolving-door jobs and money from corporations – the FDA being about the worst. Let’s say you create a huge public fight against an insurance company that has friends in government. Suddenly, emails where you revealed some odd sexual predilections surface. You lose your job because of it and are too busy surviving to continue the fight. BTW, although it’s supposed to be illegal, US military intelligence does sometimes go to bat for big companies. I found that out through experience.

All, I just want to say one thing on this topic, wait for more info to come out on this, in the next several days or couple of weeks. The way this story was presented in the article is not as simple as it seems.
Trust me, there will be more info forthcoming on this. Don’t jump to snap conclusions just yet. Please.

Probably shouldn’t raze folks about their online spelling/writing, it enflames the conversation and makes it worse for the rest of us… Please.

Sometimes folks do this out of laziness or intentionally (I know I do so at times just for humour). I like a good laugh but when it is not warranted and I see this kind of thing, well, some of us just ignore it. For youse folk observin’ – hav a gud day y’all.

USA is the only country in the world that gets involved in creating some kind of order which is very difficult with all the creeps running government’s, drug cartels, fanatic religions etc. So they have to gather data in Internet.

Dr.Richard – the constitution is a quarter millennium old piece of paper! Do you seriously think that criminals care about your constitutional rights?!? Soon technology will bring unimaginable power to the common people’s hands. Nuclear weapons?…a joke compared to what’s coming. Which of your neighbors do you trust with the “red button” that can kill everybody you know in an instant?!? The ONLY way to survive until the intelligence explosion (somewhere between 10-20 years from now, when everybody’s intelligence is enhanced enough that they are no longer posing an existential danger to others and themselves) is to YES invade privacy and prevent anyone to do harm to others, be it willingly or accidentally! Today we can 3D print a gun, tomorrow we could 3D print precision weapons and bombs, the day after we will be able to create chemical and biological agents and soon after nanobots that can invade our bodies without seeing or hearing or knowing about them and “eat” you up from the inside out…all on common people’s kitchen table.

I do understand your longing for liberty – we all want it. But where most people err is that they don’t think this coming technological explosion will be here soon or that it will be that powerful. That is a fatal mistake. To me, liberty means nothing if I’m dead. Especially that I know this invasion of privacy is temporary until with the help of technology we will all understand that some of our actions can cause irreparable damage (purposely or not, it doesn’t matter) and thus have to pass through a checkpoint which at first is set up by others until we can be trusted to set up our own checkpoints.

Sorry man, this is not a choice. This is evolution and we will pass through one way or another. I prefer keeping my biological body intact (if enhanced) until there are ways to make my consciousness “substrate independent” and that means protecting it from myself, you and everybody else any way I can. Liberty is a far 2nd on my agenda to survival!!!

The only order the US government gives for drug cartels is keeping drugs illegal and therefore profitable. Also, they give them weapons and actually participate in the drug smuggling. Yea, lets give them blanket rights to spy on everyone too while we’re at it…

No Government, once formed, wants to stay small and unintrusive. They all will rise to the level of protecting citizens from themselves but tell us its for our own good. Like wanting to limit the amout of coke we drink in a day….as if that were their business. In the name of security, you will lose your security and your basic freedom.

The core problem is secrecy. Secrecy is self corrupting and inevitably leads to abuse like this. It also makes social outcomes much more dependent on how good or bad individuals in power are and undermines the rule of law. Ever since the Cold War, maybe WW2, the US has embraced secrecy as an SOP. It is very difficult to break out of such an entrenched paradigm, and after crises like 9/11, the path of least resistance was for secrecy to explode. You cant help wondering if we could have 90% or more military effectiveness with drastically less. But it is such a huge industrial complex and military spending is so sacrosanct, it is hard to see how this isnt going to continue to be a one step back, two steps forward kind of march into the abyss. I hope Im wrong.

The other thing that is never put on the table because so many politicians and wealthy vested interests, use it as a very effective weapon, is with freedom comes risk. If you peel back these kinds of programs you will have terrorist incidents, including bad ones, there is no way around it. You can not protect a country the size of the US 100% of the time from 12 determined guys with box cutters without turning it into a police state. Even then it wouldn’t work, you would have to track every single thought, let alone email. But how effective is it, to blame the competing party or candidate, for being “soft” on terrorism, every time somebody blows themselves up in a mall.

You all know the Ben Franklin quote about People who give up freedom for security deserving neither.

This comes with the assumption that the US is in fact innocent of perpetrating these attacks onto itself as to give it a reason to galvanize it’s agenda of forcing the people into a 1984 scenario more. 9/11 being the reason they needed to take precautions or beef up security – they did 9/11 to give themselves more excuse to have authoritarian power!

Secrecy kills. If we’d known the Iraq war was totally based on lies (and oil, which we never got anyway), we might have saved a trillion dollars and thousands of lives. The secret keepers also created, inspired, and are helping to expand Al Quada. They also created our problems with Iran – all to support oil corporations. Nothing helps to control people like fear, so “terrorists” are the best friends governments ever had.

It is amusing that: eveyone kind of suspected this, that someone as seemingly altruistic as Obama didn’t see fit to make it known to his own constituents, and that when someone had the cojones to blow the whistle they are branded ‘treasonous’. AND YET, with all this information at ‘their’ disposal, drugs continue to flow like water in the streets of our cities, corporate criminals still abound, and terrorists still achieve their aims. So:

a) the system contravenes a general sense of lawfulness (illegal);
b) the government is duplicitous in its use (hidden); and
c) it doesn’t work (useless).

The last should strike a chord with taxpayers who don’t care about privacy or government dishonesty; the system is not cheap!

Since an all-knowing omnipresent machine is the clear endgame in all this, should we just be happy the US appears to be leading in the techno arms race? Should we sit back and let the Russians or Chinese take the lead?

I don’t like the idea of individual humans being able to access all info at will, but if you described to the founding fathers a magical machine that privately reads all correspondence and siphons out serious threats and notifies authorities they might actually go for it. Of course all kind of slippery slope scenarios jump to mind, but the elite break laws at least as often as anyone else so their interest in concealing their own corruption might end up protecting us all… When airborne sniffer drones start relaying the location of every bag of weed in town to the local robocops I might change my tune…

The notion of such a magical machine uses info that is not owned by it/ stolen material. Wrong is best characterized by no other word. These are the type of arguments to justify Nazi human hybrids too; should we do that too?
As for the founding fathers, I believe they would view such a machine in terms of how it might be used by those with an agenga to maintain tierany (British gov at that time). Such a machine would be repulsive to every fiber of their being.

@Bigbro:
Maybe people should not have bought into idealistic esotericism.There was never anything mystical about your “America”!

Let’s take in some more reality, to finally wake up everyone: This behavior is de facto AMERICAN, and millions of voices warned you.They still keep shouting at you to wake up.It is NOT a deviation from the usual way of things in your country.Your country is completely ordinary with respect to this, compared to other first world countries.Prism et al are old trends -present in every human society- recast into modern technology.

Let’s do away with the mysticism (I know I am grossly simplifying things, but this is my take on them):
Americans are subject to the same kind of processes as other populations.Foster the wrong cultural attitudes, give governments too much power and trust (etc), and you will see where that gets you.Or do you honestly believe the people in power are going to give voluntarily away what they have gained “recently”?The government will use what it has to expand its powers even more, everything else would be self-defeating from its perspective.

History shows this to be true, for others as well as for your own country.If you are not distracted by special periods of time in which the power of the state was swelling as a response to outside influences -like war- just to reduce somewhat thereafter, you will see that overall there has been a clear trend towards centralization/and or expanding governmental powers (in all nations, of course, dictatorships finally reach a point of equilibrium, and if other factors lead to the fall of the government our game just resets).

It is a trend which is present at all times.This even holds true for other kinds of organization and is probably due to human nature as shaped by the environment our ancestors lived in ( I feel this will be long enough already, so no going into this).

Something about beliefs and propaganda (*I guess adding some headlines makes it easier to read my stream of consciousness):
Naive belief-systems proclaiming some static, metaphysical, benevolent (self-)identity which magically stops “such things” from “happening to us” allow the “ruling classes” (for my lack of a better term) to keep accumulating “rights” until one day you will have a false awakening from your slumber, turning “your American” dream into a nightmare.

It is obvious that many people think they do not believe all the sloganeering they engage in when they scream “‘muerrica, f*** yeah” (etc) or more civilized variations, but rest assured that you cannot help but be subconsciously influenced by doing and hearing this.If you hear something often enough you will finally believe it somewhat, even if it is just to get rid of cognitive dissonance and/or due to social pressure.We humans are, if anything, an adapting animal.

Thus it seems to be the dire reality that whatever little freedom and morality we might have on this planet must be defended and re-earned again and again, at best by as many people as possible.This seems simple to the point of being self-evident, but like many simple things it is actually a very important part of life.

And like with most things which are obvious and visible to the mind (seemingly with ease), we do not act on this undeniable truth.Instead we want someone else to do it, someone like “the government” or something like the “constitution”, in the same way we’d rather eat a diet pill than exercise.

In fact, this behavior would be smart, if working diet or freedom pills actually existed (hmmm.. -freedom pills- I guess some “organization” already thought about this).

I remember someone here indulging (^^) in having a constitution from the 1787 (or sth like that); old technology that still works so wonderfully in this age… Objective ‘reality’ seems to be a matter of only minor importance to them.

Some history as it relates to propaganda and the self-perception of peoples etc:
History shows that the Americans had the best propagandists by far, and most of the time (of their existence).Even Goebbels admitted to this, he for example took techniques from Edward L. Bernays and applied them to his own country.

I must say, when I was younger I was fascinated by this conglomeration of persuasive and memetic-skill and other factors which made the common American believe so much in his people and government* (despite their proclamations of distrust for the latter- but actions and not words matter in this case), often even despite all evidence to the contrary (like the time when you invaded Iraq).
*(there have been books written by American historians about “what we falsely believe”, including the myth of a war loathing people constantly “defending” itself).

The USA showed me the kind of extremes that are possible compared to my home country, which had temporarily gone to the other extreme after losing a devastating war which was partly caused by exactly this kind of naivety, self-indulgence and extremity.

Now it is back on track and I can see many people make the same mistakes as their ancestors, but as a group we are much more wary of this than the common American, who holds strong opinions influenced by hypocritical moralizing borrowed from Christianity (which has always been hypocritical in practice- much like communism in this respect) and a lack of education (,sorry, but this is true).

Fortunately, young people everywhere seem to be more conscious with regard to politics than older people.Unfortunately, they are a minority in most (or all?) western countries.

Anyway: I do not think a totalitarian USA is going to bust free from her chains due to losing a war. The nature of this age with its WPM and interconnection will make sure that such a war will not occur.It seems more likely that other governments will feel the need to catch up.Especially since those nations who are on the rise right now are not exactly paragons of freedom either.

It is a special irony of history that the anglophone countries, which spout so much stuff about freedom, seem to be leading this development.A development which might redefine dictorships everywhere, showing what’s possible.I think we are just at the beginning of a road.A crossroad that is.

To me it seems obvious that all this is only possible because the enlightenment did not have much impact on the minds of the lower-classes (of any country).

Therefore I will end this comment with two quotations (that did not always help my country):

“None are more hopelessly enslaved than those who falsely believe they are free.”- Goethe.

“Enlightenment is man’s leaving his self-caused immaturity. Immaturity is the incapacity to use one’s intelligence without the guidance of another. Such immaturity is self-caused if it is not caused by lack of intelligence, but by lack of determination and courage to use one’s intelligence without being guided by another. Sapere Aude! Have the courage to use your own intelligence! is therefore the motto of the enlightenment…” – Kant.
* Especially important in this age of mass-media.

And finally, one that did not help your country in this “age of international terrorism”:
“They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety.” -Benjamin Franklin.

Now it is back on track and I can see many people make the same mistakes as their ancestors, but as a group we are much more wary of this than the common American, who holds strong opinions influenced by hypocritical moralizing borrowed from Christianity (which has always been hypocritical in practice- much like communism in this respect) and a lack of education (,sorry, but this is true).
Care to elaborate or are you just openly bigoted and proud of it?

Terry,
“opinions influenced by hypocritical moralizing borrowed from Christianity (which has always been hypocritical in practice- much like communism in this respect) and a lack of education (,sorry, but this is true).”