Is a wave of consolidation about to begin in the Big Data market? Take a sneak peak of #BigDataSV 2015, where Wikibon analysts, SiliconANGLE journalists and a number of top guests will debate the future of the Big Data market.

But apparently even that isn’t enough to satisfy it’s insatiable hunger for intercepted communications – our communications – which is why it’s randomly scooping up something in excess of 200 million text messages a day, according to new documents leaked by The Guardian.

The newspaper reports that the NSA has established to separate projects to gobble up and analyze our SMS messages, called “Prefer” and “Dishfire”. Further, the documents indicate that these projects are of vital importance to the NSA, with one slide dated from 2011 being titled: “SMS Text Messages: A Goldmine to Exploit”. That’s all the more surprising, since the SMS sweeps are apparently “untargeted”, which means that the spooks are basically just gobbling up whatever the hell they can get their hands on. But at least in the case of SMS’s originating from the US, the associated phone numbers are apparently “minimized” or “removed” to protect user anonymity.

We can presume that the NSA is trying to maintain a semblance of legality with this particular project, that doesn’t appear to be the case in the UK. The documents show that the NSA allows the UK’s GCHQ full access to the rest of the database, which means that it’s free to spy on messages from UK citizens. Apparently, GCHQ’s staff are told not to view any content involving UK messages, but the documents don’t say if that measure is enforced.

The Guardian reports that analysis of the content of text messages, which falls under the “Dishfire” project is extremely valuable to the NSA:

“The NSA has made extensive use of its vast text message database to extract information on people’s travel plans, contact books, financial transactions and more – including of individuals under no suspicion of illegal activity.”

As well as “Dishfire”, a second program called “Prefer” is used to scan the entire database to locate gems of information that “are not in current metadata stores and would enhance current analytics.”

And what kind of data can the NSA access from scanning SMS messages? This is surprising too – each day, it gains details of more than 800,000 financial transactions, as well as people’s travel information, gleaned from texts about their itineraries, cancellations and delays. So if you’re thinking how nice it was for the airline to warn you that your flight was delayed – well, so does the NSA ;)

The only thing that isn’t clear right now is exactly how the NSA is tapping into this data. The Guardian contacted Vodafone, but the company insisted that it doesn’t give up such data to anyone, but that doesn’t mean it’s not lying, or that the NSA hasn’t inserted some kind of ‘back door’ into its servers.

Interestingly, the NSA has responded to these latest revelations, repeating its usual claim that its activities are only focused on “valid foreign intelligence targets in response to intelligence requirements”. Like we’re supposed to believe that “valid targets” typically send in the region of 200 million SMS’s each day…

About Mike Wheatley

Mike Wheatley is a senior staff writer at SiliconANGLE. He loves to write about Big Data and the Internet of Things, and explore how these technologies are evolving and helping businesses to become more agile.
Before joining SiliconANGLE, Mike was an editor at Argophilia Travel News, an occassional contributer to The Epoch Times, and has also dabbled in SEO and social media marketing. He usually bases himself in Bangkok, Thailand, though he can often be found roaming through the jungles or chilling on a beach.
Got a news story or tip? Email Mike@SiliconANGLE.com.