Barack Obomba and his minions have claimed that the metadata that their PRISM program is entirely benign as only certain information has tracked and recorded on millions upon millions of Americans.As though offering proof of that contention, they have said that they aren’t recording conversations, as though that would be the only actual intrusion into our privacy, and for recordings and transcripts, they would need warrants. Few warrants have ever been denied. But being ignorant of what these two different kinds of intrusions in fact entail, this interview was very helpful to my understanding, and I reckon that it might be for you, as well. The rush transcript is now online.

Also from Democracy Now this morning: Is Edward Snowden a Hero? A Debate with Journalist Chris Hedges & Law Scholar Geoffrey Stone; it’s a bit long, but Hedges makes it clear what’s at stake, and has distilled his talking points well. The rush transcripts aren’t available yet; I’ll paste in the links once they’re available. The rush transcript is now available.

Earlier today the Guardian reported that Snowden had met for an interview with the South China Morning Pressin which he said he intends to “he intends to “ask the courts and people of Hong Kong to decide my fate,” and was about to reveal new information.Interestingly, the SCMP website’s servers are down now.Hopefully they’ll be up again soon; journalist Lana Lam had promised an hour or two ago to have the full interview up soon. From Lam:

The ex-CIA analyst has been holed up in secret locations in Hong Kong since fleeing Hawaii ahead of highly sensitive leaks revealing details of US top-secret phone and internet surveillance of its citizens.

the steps he claims the US has taken since he broke cover in Hong Kong

his fears for his family

Dave Lindorff, who speaks Chinese and lived in China for several years, has an interesting take on ‘Why did Edward Snowden go to Hong Kong?’, including the fact that he did a great favor going public the day before Obomba met with Chinese President Xi Linping in California. His reasoning is in direct opposition to Ken Klippenstein’s ‘Greenwald Botches NSA Leak’, which blames Greenwald for the danger that Edward Snowden faces in Hong Kong, and seems to believe that Glenn could have steered him toward a country in Latin America.

In case you’re interested in what one Democratic Senator from Minnesota had to say about recent revelations on all of this, Al Franken said that he knew all along this was going on, and:

“I can assure you, this is not about spying on the American people,” Franken said.

Franken, who sits on the Senate Judiciary Committee, says he got secret security briefings on the program and he says it prevented unspecified terrorist acts.

“I have a high level of confidence that this is used to protect us and I know that it has been successful in preventing terrorism,” Franken said. (It just struck me funny.)

Added: The Guardian has developed a tool that they say will allow you to discover what the metadata looks like for the services that you use. The page shows an example, and then shows what could be surmised in the conversations between David Petraeus and Paula Broadwell. Heh.

(‘Release the Hounds!’ by anthony freda, via wendydavis @flickr.com)

Stay safe, Edward Snowden. What an incredible service you’ve done for us, as have Bradley Manning, Thomas Drake, Julian Assange, and many other whistleblowers whose consciences would not allow them to remain silent. And thank you, as well, Glenn Greenwald, for your dedication to the truth and the US Constitution and Bill of Rights.

133 Responses to Is Metadata More Intrusive Than Eavesdropping?

Can’t link to a youtube, cause I’m at work and the filter won’t let me search there, so I’ll go off of memory (I doubt you’re a big fan of Judas Priest, but the words were extremely prescient in 1982).

Up here in space
I’m looking down on you
my lasers trace
everything you do

You think you’ve private lives
think nothing of the kind
there is no true escape
I’m watching all the time

I’m made of metal
my circuits gleam
I am perpetual
I keep the country clean

Brrrr. But now so many have the data who are just…private contractors.
Gonna be interesting to see which ways this all falls.

Can’t wait for Edward Snowden’s new targets info, either. The SCMP was down for a few hours this mornin’, but it’s back up for now. Server overload?

Thank you for the lyrics and song. Yah; it’s better, and thank you, darlin’. I did a nice 35 minute meditation video that I like earlier, so HA is on the mend. So I did listen, and you pegged it: I don’t know if I’d ever even heard of Judas Priest, the band. Know Dylan’s song of the same name… ;~)

Yes, Senator Franken, author of a book about U.S. politics entitled Lies and the Lying Liars Who Tell Them.

Like you, I have 100% confidence that no information that you got from the Executive Branch to justify this program (or any program in which the Executive Branch wished to engage) was the truth, the whole truth and nothing but the truth.

/s

Thanks, Wendy. Whether or not this program is more intrusive than listening to my conversations or reading my emails, it’s too intrusive. That’s all I need to know.

Hard to disagree with that, ncbb; but there is a need to call bullshit on Obomba’s phony lies. Which reminds me, the new term ‘telephony’ (as in that sort of data, makes me cringe. We need new words besides the existing ones to stand for the mind-reeling surveillance and propaganda we’re living with, and under, now.

Both interviews provide good talking points refutations to the LOTE folks, as well as our neighbors who claim that it’s all benign, and being done to keep us safe from Our Enemies.

Franken: yes; an extra hilarious kick given his book and all. Of course, maybe he was just doin’ a bit of comedy, eh? Offering some golden satire?

We’ve known for quite some time that once elected, seemingly good guys turn into ‘them’. Rubbing shoulders with the True Elites and Powerful must create some exotic and heavy perfume of: ‘I Want That!’

Yes; just imagine, blueokie. This G has nothin’ on any other totalitarian regime except more of it.

Thank you for bringing up this question, wendydavis. I found it most helpful to watch the Susan Landau piece after reading the transcript of the debate. The latter brought up for me a question as to why Geoffrey Stone felt he didn’t have the information with which to call Snowden a hero. It appeared he took the matter of legality at face value – ‘a law is a law is a law’ with no apparent consideration that matters of conscience could be involved, (or even the Bill of Rights). Early on Chris Hedges makes the point

“…we are at a situation now, and I speak as a former investigative reporter for The New York Times, by which any investigation into the inner workings of government has become impossible…”

What then becomes of the freedom of the press enshrined in the Constitution?

Susan Landau, in addition to pointing out the ways content can be surmised from metadata, (along with identity and habits), emphasizes that secret legal interpretations of law are really what has been the matter of conscience for Snowden – he is outing that secretly metadata is being interpreted as outside the bounds of the restrictions (wet noodles though they be) already in place. He, Snowden, says correctly (and heroically) that this is something we have to know about.

This is just exactly like the Bush stuff about waterboarding being enhanced interrogation and so they do it hundreds of times on one person (and never get impeached for doing this). But it is even worse because now it is a secret interpretation. Bush, to his “credit” released the memos supposedly justifying torture. Obama prosecutes the hero who releases the information.

Consider my comment neutral. Most if not all of the so-called metadata is transmitted and received wirelessly once it leaves the transmitting device (phone or PC). It travels as RF signals or microwaves, so it’s all out there in the free air. Anyone can capture it, so the question I’m waiting to have answered (if the ACLU suit goes anywhere) is Why should the government be prohibited from capturing it (meaning, like listening to the radio) when no one else is prohibited?

Okay, now a bias: At least FDL doesn’t require everyone to pee into a cup if they want to post here.

AitchD, can “anyone” capture ALL of it, comprehensively, for years, put it into storage, retrieve it based on filters, and compile lists and profiles and generate reports with it catagorizing and implicating entire groups of people and their affiliations?

Oh, and I almost forgot, NOT SECURE IT from the TRUSTED PROFFESSIONALS whoever the fuck they are, who design, build, manage, maintain, wield, and install back doors that ignorant/stupid bureaucratic fucks have no ability/desire/concern to see in it?

Interesting. juliania, especially given that you watched Hedges and Stone first. Not being a particular fan of Hedges (a plain heresy), I dove for the explanation Landau brought, since I am pretty tech clueless. You brought some of the most pertinent quotes.

I loved Landau’s: “”What is he talking about in Section 215?” admission. Veddy.Secret.Interpretation.

Your Bush v. Obomba extrapolation is also notworthy; I hadn’t thought of it that way. But we’re still waiting for his DOJ’s explanation of why drone assassinations are…legal. He twists the word ‘legal’ to mean acceptable and Constitutional, and most sadly buy it. Seeing his ‘base’ not only blithely accept it, but *defend it*, rankles.

There is something subtle in what’s going on that is still not quite dawning on everyone. Clapper’s muddled attempt at an explanation offers proof of something I’ve suspected for quite a while.

The new doctrine is that “big data collections” are considered facilities. Not sure why that verbiage is important, but it appears to be. The interpretation appears to be that the act of *gathering* information into a facility isn’t surveillance *if they don’t look at it* while it’s being gathered.

Under this theory, you have not been placed under “surveillance” unless an authority withdraws information about you from one of these facilities. Clapper ran into trouble by trying to replace the word “surveillance” with the word “collection” in his tortured Dewey Decimal metaphor. Which kind of highlights why the basis could very well crumble if subjected to critical scrutiny.

Further, it appears as if creating a piece of software to automatically withdraw information and perform relational analysis also does not count as “surveillance” … maybe by virtue of records not being “looked at” by a human despite being taken off the shelf and assessed? This is certainly the case in the meta-data database.

The president was very careful to limit his assurances that nobody was listening to American’s phone calls to the specific program being discussed. But there is a huge question left hanging here.

We know the phone surveillance disclosed is being used to create a facility for phone Meta-Data. We know there is another one creating a database of internet meta-data; that may be BLARNEY – or BLARNEY may be something else. We know these facilities are compartmentalized and that many others are also being created for other types of data; I think credit card transactions may have been confirmed.

My understanding of the doctrine is that technically *any* data that could be obtained would theoretically not count as “surveillance” until it was accessed. There does not appear to be anything to keep them from creating a facility for anything – like all conversations in America – as a compartmentalized capability and providing the same query-based ability to perform retroactive “surveillance” on any data in the facility. It would simply have a different name.

I don’t remember the exact quote, but the “ah-ha” moment over this came a couple of years back when Marcy was digging into exigent letters; and the phrasing made it sound like there was access conversations that occurred prior to the demand letter being issued which could be recovered retroactively. I swear the term used that piqued my closer scrutiny was “facility” … which made no sense in the context of a wire-tap. At the time, Marcy said I was crazy and there was no basis to speculate (or maybe it was William Occam).

So far, we are still asking the wrong questions. But it’s getting closer. I think the essential question is how to imagine it is possible to split the concept of collecting data on a person from the definition of conducting surveillance on them without making a mockery of the 4th Amendment.

Worrying about the features of how our national security apparatus interacts with a particular facility might be missing the forest for the trees. The basic premise behind *all* of them is flawed.

To your first point, I (for one) am grateful that only a federal government has such a capacity. Within your point is the expectation that the process is mainly unethical and approaches being evil in intent. Of course any system is corruptible, even if it’s entirely automated; even if no eyes ever look, it will fail inevitably, being the nature of complex systems. I hope you’re wrong. Unless you’re right, I think it’s okay to have a global, universal stakeout rather than an inefficient one based on unverifiable tips from disgruntled and bitter folks (Obama’s words).

As for examples of metadata being intrusive, Susan Landau gives some good ones in the DN piece, but an ACLU attorney on France24 the night before last had some that were even more direct: calling a suicide hotline and calling an abortion clinic (there was a third, but I’ve forgotten it).

When you’re back from your nap could you perhaps tell us where to find the articles by Dave Lindorff and Ken Klippenstein?

- meta data can’t hurt ya
- that ‘ya’ll needs a warrant’ is bullshit in a world of corporate cooperation with despotic government insiders (otherwise defined as fascism)
- nothing to worry about if yer a patriot
and…
always good to hear the cogent clarity of both hedges (an american hero) and lindorff
and…
A.Franken has a monumental blind spot when it comes to counter-terrorism/rights abatement programs targeted at ‘the axis of evil’

and … always humbled, ms.davis when reminded of the heart behind those typing fingers >peace

The information has come so fast and furious that it is easy to conflate what we know. And that is:
1. A court order from the FISA Court under Section 215 of the PATRIOT Act as amended several times authorizes the routine delivery of telecommunications connection metadata (the court orders lists the particular data fields delivered) from telecommunications providers. This order covered all telecommunications handled by Verizon Business Systems, both international and domestic. DiFi says that this is a routine (i.e. standing) order reissued periodically and most reporting assumes that it applies to all telecommunications carriers within the reach of a US court order. And the recipients are both gagged and immunized against lawsuits from their customers. Among the customers of Verizon Business Systems are the ACLU, the NYCLU, and (ironically) the Supreme Court. When you hear about “metadata”, think of this particular piece. And remember that it is more than metadata of phone calls; it is metadata of text messages and data accesses to and through the internet.

2. From past reporting, telecommunications companies have separate NSA rooms that receive copied information from the stream coming into the telecom central office switches. The equipment that likely is used for this is hardware from a company called NARUS. So the stream of metadata that the FISA court order pretends is coming in periodic batches is really coming in in real time through these rooms that were the original tip-off that this was going on.

3. There exists a system at NSA called PRISM that under some as yet unknown legal authority and through as yet unknown technical mechanism (although Palantir’s Prism application has been mention and Palantir has denied it). This is a real-time system supposedly sucking out content through some virtual equivalent of “direct access” that protects from the third-party companies knowing what NSA is getting. This is not metadata, although it likely includes metadata. And it is “phone calls” only if you use Skype or a similar service over the internet.

So there is no “direct access”, no “listening to phone calls”, and “only the metadata”–all technically true and all misleading. And all “legally obtained”–that is, under the color of law.

Is the metadata that is received more intrusive than eavesdropping? Well that depends on who you are and what you do. If you only use your iPhone to access Pandora, they know you listen to music but not which genres or songs. If you are the person with cancer referred to in the Democracy Now report, it is very intrusive and depending on the specialty of the surgeon might be able to distinguish between whether you have brain, breast, ovarian, or liver cancer. If you are a patient of a mental health professional with a specific specialty, it is even more intrusive.

And then the cell-tower triangulation (for 911 purposes, remember) saves slamming a GPS tracker under your car. That not just intrusive, that is stalking intrusive.

So what are some of the pertinent trust-and-verify questions that need to be asked.
–Has the FBI requested searches from this data for suspects for whom there is no probable cause of their being terrorists? i.e. immigration law violation, drug cases, unsolved crimes, and other searches of convenience.

–Was information from NSA used to track Occupy Wall Street? This is the biggie, as big as the COINTELPRO operations against activists that were exposed by the Church Committee.

–What are the safeguards that prevent innocent people from being identified and treated as suspects through the correlations in these “big data” searches? (that is, how are the searches minimized to screen out seeming matches that result for scanning large amounts of essentially dirty — uncorrected for spelling, duplicates, or data field transposes [smith, ralph instead of ralph, smith])?

–Has the data been used to identify potential informants for any law enforcement agencies? (an really spiffy use of bad correlations–we’ve seen this principle used before to create a narrrative)

– Since 2001, how many employees have been fired for security violations, and what exactly was the nature of what they did that caused them to be fired?

– What are the “legitimate channel” whisleblowing procedures? How many cases have occurred since 2001? What were the accusations made in those cases? What was the disposition?

– How are voice communications or film communications transcribed to text if they are or otherwise referred for further analysis? Note this is a chain-of-custody question, not a criteria for referral question.

– Does NSA share data about US citizens with private corporations, other organizations, or foreign governments?

– How does a US or foreign citizen misidentified as a terrorist suspect by NSA go about clearing their name?

There is no reason in my mind for preventing the answers to all of these questions from becoming public. They compromise no sources (we know those already–every third party you deal with) or methods.

For a Commission there are other questions that would likely be kept secret. What other systems are there and what is the scope of the data they receive–US citizens? (should be off limits), international third parties,…

What are the agreements that NSA has with foreign government intelligence agencies and do the foreign government’s heads of state know about those agreements?

I’d seen a long comment you’d made on (iirc) the ‘willy-nilly’ Obomba/Clapper thread on the front page the other day, and you lost me rather quickly. This is easier to understand, at least a bit, so: thank you.

It hadn’t occurred to me that ‘data collection’ and ‘facilities’ were being conflated, but I had noticed that the art used in most pieces was either the site in Maryland or the one in Bluff, and you might explain the destinations for us, if you would.

With a nod of appreciation to the sly terms they’re covering themselves with, I’d thought that the phone conversations *were* being stored (dunno since 2007 included), and given their *enough evidence for a warrant*, they could/would then listen to and transcribe the actual voices (but we know the FISA court’s only rejected two applications over the past four years or so, no?)

Is the implied ‘basis for a warrant’ this that you said?

“Further, it appears as if creating a piece of software to automatically withdraw information and perform relational analysis also does not count as “surveillance” … maybe by virtue of records not being “looked at” by a human despite being taken off the shelf and assessed? This is certainly the case in the meta-data database.”

And what is BLARNEY? And…I do hope others catch your drift better than I probably am. I know you are a tech wizard, and can teach us stuff that’s being missed. You know how clueless I am about all this.

Also, as I was trying to let kgb’s comment soak in, and before reading it a third time, I’d been wondering about the speculation made by some folks that the program simply taps directly into the servers of all the services mentioned. So I wondered if that may have been one of the things Snowden was about to bring.

Anyhoo, the longer piece promised by Lana Lam at SCMP is up, and he’s told her that the CIA and ISG has been hacking into computers in Hong Kong and the mainland for years:

“US National Security Agency’s controversial Prism programme extends to people and institutions in Hong Kong and mainland China;

The US is exerting “bullying’’ diplomatic pressure on Hong Kong to extradite him;

Hong Kong’s rule of law will protect him from the US;

He is in constant fear for his own safety and that of his family. [snip]

Snowden said that according to unverified documents seen by the Post, the NSA had been hacking computers in Hong Kong and on the mainland since 2009. None of the documents revealed any information about Chinese military systems, he said.

One of the targets in the SAR, according to Snowden, was Chinese University and public officials, businesses and students in the city. The documents also point to hacking activity by the NSA against mainland targets.

Snowden believed there had been more than 61,000 NSA hacking operations globally, with hundreds of targets in Hong Kong and on the mainland.

“We hack network backbones – like huge internet routers, basically – that give us access to the communications of hundreds of thousands of computers without having to hack every single one,” he said.

It remains to be seen (adjudicated?) if the dragnet program violates the 4th Amendment. The personal user TOS agreements with carriers and software licenses pretty much require the user to waive 4th Amendment recourse as well as some other Constitutional guarantees.

I’ve posted elsewhere about a worst-case false-positive scenario: I found a woman’s cell phone on my mother’s chair after the woman left with her daughter. I turned it on (no password required), scrolled the contacts and located the daughter’s cell # and called it. The daughter answered and instantly realized from her caller ID what was now self-evident. They returned and got the misplaced phone. But what if the mother had been a targeted enemy combatant, and my using her phone alerted the nearby drone’s command center of the present location, locked in, and fired a missile that killed me and my mother? I mean, my mother was innocent.

Up @11 I mentioned peeing into a cup as a condition for participation. Back in the 1980′s when anyone who wanted a ‘good’ job had to pee into a cup as a condition of employment (but only after a valid test for THC had been developed), it became clear that only a fringe (of many ideological threads) gave a crap about the 4th Amendment. Personally, I don’t know many people who have had to pee into a cup, but I’m sure once I got to know them, I’d wish I didn’t know them.

We let private law (TOS agreements) go unchallenged, we let cup piss go unchallenged (that Yuppie gen could have just said no). It’s no wonder we have the kind of wretched government we have to match the wretched corporate reign.

They are hacking foreign telecommunications servers and offices for the same information they are getting through court orders in the US.

kgb999′s comments about “facilities” is interesting. It is likely a within-the-beltway buzz word or a term of art for large storage management systems.

The cute argument that they are making is that taking your data from third parties does not constitute Fourth Amendment seizure. That only when they view it does it become a legal issue and then it is under the color of law as a FISA-court warranted Fourth Amendment search. I think that sketchy logic is what has a lot of the legal beagles buffaloed.

Oh, crikey, THD. This one’s gonna take some time to digest; just the ars technica piece alone looks fascinating, but hard for a Luddite like me grasp very well. Oopsie, I should have read it before I answered about OWS and the banks. But yes, we’d have to believe so, yes? More soon as I can walk through it. And others will likely have comments on it. Great to have all that here for posterity, as well; thank you.

After I’ve been gone for a bit, I have a bad habit of scrolling to the end of a comment stream. Our daughter called to report on some large fires north of Colorado Springs; Black Forest has been evacuated, and a couple other smaller towns. High winds last night.

Just speculating b/c I don’t have the kind of security clearance to know anything. Nevertheless: Twenty or more years ago during a visit to Pgh by our mother, my sister wanted to go to “a new Chinese restaurant that got a good writeup”, while I wanted to go to a tried and true one. We went to the new one my sister wanted to try. I don’t recall liking anything about it, but of course I was already prejudiced. Ten days later the restaurant was closed, its owner arrested, after dozens of dead, skinned cats were found in the cellar hanging from a wire.

You might paste this comment in ubiquitously on front page pieces, THD. There were a couple other good ones that readers asked Noah Sachtman yesterday on his live Q and A session, but, as ever…I’ve forgotten them. And I can’t find the piece there now.

Same for kgb’s comments; they deserve to be seen, and others here as well. But not AitchD’s about skinned cats, lol!

Don’t you think if all this I-Am-The-New-Daniel-Schoor breakneck disclosure stuff were genuinely harmful to the intel group and gov’t, there’d have been an Internet ‘curfew’ by now?

I’m skeptical with no nose for news. Until I learn otherwise, I’m riding on the very plausible theory that Petraeus had plotted a Seven-Days-In-May coup if Romney lost. (And I think Obama’s forces counter-jimmied the electronic voting machines.) He was found out early on. Many heads are targeted to roll from that, but it has to happen gradually and most carefully. I’m guessing when the scripted Congressional hearings take place, it will come out, being the only public accounting we’ll get about all this secret intel stuff. Snowden may be a good-guy plant whose ‘real’ job was to nudge the snowball. As a bonus, we’ll get some privacy protections written into law.

Mahalo, wendy, for another excellent post…! One thing I’ve not seen discussed yet, is how the FISC Court is set up, such that all the Judges are selected/appointed by the Chief Justice of Scotus…! Any wonder why there’s been no denied ‘warrants’…? 8-(

But sufferin’ succotash, Perfesser; I wish you would have offered that theory when Larue was beggin’ for ideas as to what was at the bottom of that.

I can only hope that will come out sometime; it will be too delicious. But oy, it’s been too many decades since I read ‘Seven Days in May’. Knebel, right? I think I have a first downstairs with a crappy dustjacket. Want it if I still do?

Ah, the post was mainly a PSA, but welcome. And I figured correctly that we might learn more from the comment stream, as always. And lookit: you didn’t disappoint: I did not know that the Chief Justice made the assignments. Holy hell. Yes, small wonder.

…In other words, Americans seem to be fine about any kind of U.S. government spying as long as it’s focused on “foreigners.” They just go crazy when the same tactics are used on Americans.

That’s odd. And a bit disturbing.

Because most people in the world happen to “foreigners.” And most of these foreigners are as non-violent, law-abiding, and deserving of basic privacy rights and due process as most Americans.

Some of these “foreigners,” in fact, are friends, family, colleagues, and customers of Americans. And most of the billions of users of Facebook, Google, Yahoo, and other major communications service providers whose communications the NSA is “intercepting” are foreigners.

There about ~7 billion “foreigners” in the world, in fact, and the U.S.’s Foreign Intelligence Surveillance Act apparently gives the U.S. government complete freedom to spy on all them…

It’s the idea sold to justify Guantanamo that the Bill of Rights only protects citizens of the USA. And that the USA has no obligations to obey human rights conventions that violate US sovereignty. You remember that BushCo argument, don’t you? Apparently it still is around. American exceptionalism and all that.

In a Wednesday letter addressed to the Privacy and Civil Liberties Oversight Board, Sens. Tom Udall (D-NM) and Lisa Murkowski (R-AK), among others, request the board examine whether the NSA data and phone record collection programs are legal as authorized by Congress and whether they adequately protect the privacy of millions of Americans.

Obviously, Americans are easily manipulated by Propaganda and by that old trick from Hermann Göring: Tell them they are being attacked.

Pew Charitable Trusts are a Propaganda organ of Shell Oil. When Pew’s BS gets the deepest is when it is incarnated at the Pew Climate Trust, very often on C-SPAN as an expert, perhaps to represent the Left – the Left end of the family that controls Shell.

““A nation can survive its fools, and even the ambitious. But it cannot survive treason from within. An enemy at the gates is less formidable, for he is known and carries his banner openly. But the traitor moves amongst those within the gate freely, his sly whispers rustling through all the alleys, heard in the very halls of government itself. For the traitor appears not a traitor; he speaks in accents familiar to his victims, and he wears their face and their arguments, he appeals to the baseness that lies deep in the hearts of all men. He rots the soul of a nation, he works secretly and unknown in the night to undermine the pillars of the city, he infects the body politic so that it can no longer resist. A murderer is less to fear. The traitor is the plague.”

A great comment from a diary on dKos; you’ve got to read it all the way through for it to hit you.

As a software developer … (127+ / 0-)

…I’m envisioning the reality. More and more ideas of what data we can gather. Each requiring an IT project. More and more neat ideas of how we can use this data, each requiring an IT project. Trying to mine the data, but it’s messy data, and you have to make assumptions, and what comes out is sort of what you had in mind, but not quite… Never enough money to do it all. Budget cuts. An IT department that is falling behind, with yesterday’s technology, drowning in a deluge of more and more and more data. The little hamsters in the computers, racing on their wheels, trying to dig through the data, faster and faster and faster, but it’s never fast enough, while the users complain that they can’t do their jobs, because they can’t get the right information, and it takes too long to get whatever they can get, because there are all these stupid rules,, and when they do get it, it’s not right, and it’s not enough, and it’s too late, and the people who DID have the information didn’t bother to share it. And, you can’t even use two thirds of it, because it’s not cross-referenced properly.

We need to replace the entire system, now, because it’s fifteen years old and it’s about to die because the software vendors stopped supporting their versions of the software, and it’s not practical to upgrade because it’s too customized, and it’s incompatible with the new release of Windows, or Oracle, or whatever. And, it’s like ancient, some old touchscreen technology, when everyone is using holographic projection now. But, it’s so huge. And, so expensive. And, it’s going to cost twenty times more to replace it than it did to develop it fifteen years ago, (more, if you count the cost of replacing everyone’s desktop with holoprojectors) and it’s going to take ten years to get it done, at which point it will launch with already outdated versions of software and we’ll have it all to do all over again And, besides, we don’t just want what we had before. We want it bigger, and better, and newer and shinier, with heated seats, and a backup camera, and lane assist.

And, then you can double that cost, because we also have to build a second system for off-site disaster recovery, because the world would come to an end if we had a disaster and the GiantDataWarehouse went offline. Someone might buy some pantyhose, and we’d never know about it!!!

Look at how hard it was to just re-write the air traffic controller’s system, or to implement a new baggage handling system in a major airport. These projects are like trying to build a rocket to go to the moon.

And, then we are back in the real world. Some evil persons blow up the White House, like 911, and we have a huge investigation into why, with ALL of that money we spent, ALL of that data at our disposal, we couldn’t manage to find the one piece of data that would have stopped the disaster before it happened, when the piece of data is RIGHT THERE, and we’re all standing here looking at it, thinking that if we had found it just a few weeks earlier, everything would have been different. But, we couldn’t find that one little needle, and the haystack is growing exponentially larger, bigger and bigger and bigger and bigger, making the odds that any future needles ever could be found, smaller and smaller and smaller…

Meanwhile, some politicians have made friends with the analysts, and are using the fabulous GiantDataWarehouse to spy on political opponents, and gain insider information to make piles of money, incidentally causing a stock market crash, and plant false information to sabotage their enemies.

And, somewhere, one person’s data got accidentally linked up with another person’s data, and his life is ruined, because this ordinary guy from Hoboken is being confused with an evil Pastafarian Terrorist. He can’t get a job, and no one will rent to him, and he’s not allowed to fly, and he can’t buy anything because there’s a new law that no one can take cash, because you can’t trace cash transactions in the GiantDataWarehouse, and the cops keep harassing him.

I am very concerned about my privacy, but I also think there’s a significant danger that the government might simply suffocate itself under it’s own weight.

I wouldn’t expect any more than a half measure from some of the names on that list. Is that who is left to “defend” Civil Liberties, those 13 Senators?!

Four of those names particularly irk me today: Murkowski the Oil lady; Coons the climate-destroying chemical pollution magnate who got installed in the most disgraceful rigged election in modern memory; Max Baucus who very publicly accepted Bribes from Big Pharma and Big Insurance to sell out the American people on healthcare; and my own Senator, Elizabeth Warren, who campaigned pushing for war with Iran.

When you see that happening, you’ll know that the the worker bees have unity. Wonder what safeguards are in place to forestall that possibility. Snowden was under the impression that all folks’ data was equal. Might some be more equal than others?

Man, Larry O’D had a clip of Barbara Mikulski bemoaning the fact ‘that just because you’re an Olympian swimmer doesn’t mean you make them a Navy Seal’…! Didn’t she get the memo that they’d actually sent him to Q-school…? ;-)

Mikulski is protecting jobs in Maryland. She is having the same problem with a good rhetorical argument that all of the defenders of the status quo are. The evidence is clear and the government position is fundamentally indefensible (except by STFU force).

Somewhere I saw -must have been a nakedcapitalism link as I’ve been there all afternoon – those folks are really freaked out – I guess the more cellphones you have, and all of that. But maybe there I saw a link to Congressional updating by our leaders (strictly secret of course) and one lady congressperson by the name of Sanchez said that we’d only seen the tip of the iceberg, but of course she wasn’t at liberty to divulge more (not as brave as our whistleblowers). Do remember this anyone who might be inclined to vote for her in future.

Also, another piece I can’t exactly place – just as well as I don’t know if it was snark or not – Australia is purported to be building its own mega storage facility just outside Canberra for US storage of data,(nice of them if true) so I guess it is like the something-something array they are always telling us about for probing the secrets of the universe…poetic, that.

I also wanted to play devil’s advocate here, and there’s a blogger who has been linked to – Arthur Silber? – various places who is totally irate and thinks all of this is playing us for fools, dangling out little bits and pieces, being careful not to irk the PTB. His pieces are thought provoking, though I’d be more irked had we not got any of this to chew over. Something is better than nothing. Can be googled, I’m sure – I just couldn’t find his blog as I was typing this.

Great minds think alike I guess – I was already thinking about that moneymaking thingie and trades and all of that, who emails whom when – mighty interesting to have the finger on that button. Oh the possibilities are endless, I tell you, endless!

Arthur, I really can’t think our leaders are happy about all of this getting spilled, even in dribs and drabs. I really can’t think that.

“We hack network backbones – like huge internet routers, basically – that give us access to the communications of hundreds of thousands of computers without having to hack every single one,” he said.

… essentially, is the idea behind BLARNEY – although they seem to have this easter-egg split up under a couple of different named programs. Perhaps something derived from hacking a foreign network would have it’s own name.

BLARNEY is a component in the NSA “Upstream” capacity – as detailed in this slide. There is also a program called FAIRVIEW and two others with the names blacked out.

Description of BLARNEY from the Washington Post article:

The Silicon Valley operation works alongside a parallel program, code-named BLARNEY, that gathers up “metadata” — technical information about communications traffic and network devices — as it streams past choke points along the backbone of the Internet. BLARNEY’s top-secret program summary, set down in the slides alongside a cartoon insignia of a shamrock and a leprechaun hat, describes it as “an ongoing collection program that leverages IC [intelligence community] and commercial partnerships to gain access and exploit foreign intelligence obtained from global networks.”

Essentially, BLARNEY does for the internet what the Verizon phone records subpoena does for phone traffic. The description sounds like it is based on the NSA/AT&T joint program disclosed by Mark Klein almost 10 years ago.

In the WaPo article, they are talking about metadata – but that pipe AT&T provided feeds the NSA everything. The meta-data is simply the part of the data used by the system to get information from one place to another and providing a general description of the type of data being moved.

It would be possible to dump the meta-data in one database and call it BLARNEY and dump the content in another and call it FAIRVIEW … and then as long as they could construe a question from congress as being about BLARNEY, they deny gathering American’s content until the cows come home – even though it’s sitting right there in essentially the same database.

Also notice that the NSA slide I linked appears to indicate that the overall Upstream capability would provide access to a collection of communications – not just meta-data.

One other thing that most people may not realize is that once communications are traveling across the backbone, there is literally no distinction between voice and internet traffic. It all flows down the exact same cables. That AT&T optical feed really could copy pretty much everything – both voice and data that AT&T handles; it’s not just an internet tap.

It’s fascinating how many dots just a few slides bring together … and annoying how many holes remain.

This PRISM program is not at all like the Palantir software. Palantir’s software does relationship-mapping. The easiest way to view it would be that Palantir could be fed the social networking data returned by PRISM to perform deeper analysis.

At it’s core PRISM really appears to be a simple automation tool – likely module based. It would start with a warrant (or whatever) and then produce the customized information demands for each ISP, watch for the data to be returned to the correct sftp directory in the correct format (the “drop-box” Google denies using), and then alert the agent when their information is ready. In and of itself, it seems the least-worrying aspect of this; it’s their front-end for serving warrants. If they didn’t have the tool, the demands would all be processed by hand … just they do now for the uncooperative Twitter or any ISPs not involved with the program.

If I were speculating, I’ll bet what they were doing to get slapped down by the FISA court was linking the part of their system that trolls data looking for “terrorist connections” directly to the part of their system that automatically certifies and produces NSL demands. I’m guessing their first approach was to have the system algorithmically identify suspicious records and then, when one was found “51% likely to be related to a foreign terrorist,” the NSL the demand letters would go out and ultimately a complete dossier on the person would show up on an analyst’s desk for review. I’m guessing the court ruled that an actual human needs to issue a Section 207 demand in order for it to be constitutional.

Also. It seems to me to be impossible to get from the Verizon phone data to a list of social networking and email accounts and back. Alexander’s senate testimony today really just made stuff more difficult for themselves – how can one check a phone-record database to see if a Twitter user is likely in a foreign country?

The part of the system that we know least about is the part Alexander alludes to which runs queries on the meta-data and gives them a list of Americans to spy on.

While it’s hard for me to grasp the particulars, especially following the links, it seems that this is the core thing to have learned:

‘…once communications are traveling across the backbone, there is literally no distinction between voice and internet traffic. It all flows down the exact same cables. That AT&T optical feed really could copy pretty much everything – both voice and data that AT&T handles; it’s not just an internet tap.’

Thank you; I’ll keep trying to let it soak in. I got sidetracked trying to recall other early secret communications surveillance whistleblowers’ names since Klein’s name didn’t ring the bell it should have, and meandered all over this morning. Several names I tried led back to this piece at Washington’s blog. Pretty much the same title as the one DN interview. But the links go to old reports, Israel-designed programs, tra la la. So many acronyms, so many secret programs.

The good news is that many folks on the thread will grok what you’ve written far better than I, lol.

And dividing the same data into different program names is great a great way to maintain deniability…until it’s not.

Did you read the Ars Technica article on searches. It looks like a tagged search process instead of a structured relational database with pre-defined columns. The metadata and likely source and datetimestamp becomes the initial set of tags on the content as it is stored by NSA. So you can minimize to the relationships shown by the metadata initially and prioritize additional tagging based on those relationships.

The large storage facility now makes sense. When you are building essentially a mirror of the entire dynamic global internet, you need a lot of storage. The static sites you can hack directly. So you are not mirroring the entire internet, just the most changeable parts that function as communications traffic.

Why the NSA folks are so outraged is that every other government in the world now knows for sure that NSA wants everything. And Angela Merkel is already not a happy camper; she thought that NSA was minimizing just to terrorists but now she suspects that NSA has her email too.

The (furious) Arthur Silber piece that juliania was likely referring to is chock full of goodies some of you might want to weigh in on. Basically he’s outraged that Snowden, Greenwald, and Gellman (with some host of attorneys) have decided that much information, and many of the PRISM slides were withheld.

He’s pissed that they use terms like ‘could/would cause harm’, ‘public value’,'journalistic standards’, and others that they never define in any sense, which Arthur concludes…thus, they mean absolutely nothing. He sees them acting not as revolutionaries, but as a different level of gatekeepers to the security state, but those are my words, not his.

“In summary: we are left with terms that are never defined and/or that are largely devoid of meaning, and a process of reaching judgments which remains a complete mystery to all the rest of us. That is to say: we have absolutely no way of evaluating what it is they are doing, for the simple reason that we don’t know what they’re doing with any specificity. And in terms of what has not been disclosed, and as I already pointed out, we are not given even the slightest indication of why particular information has been withheld. With regard to the 41 slides that make up the Prism “brief,” for example, we might have been told: “Slides 9 through 16 concern X,” where X is still generally described, if necessary, as for example: “disabling computer security systems in terms we view as potentially helpful to enemies of the U.S.” That at least would make what they’re doing minimally intelligible. But we’re provided no details at all. We just have to “trust” them. Do you hear those alarms going off again? You should. [snip]

(a long section of elitism, trust, yada, yada…)

“I’m especially not buying it because the State’s insistence on “secrecy” is a series of lies from beginning to end. And I implore you always to remember that we are speaking of a State which arrogates to itself the power to murder anyone it wishes, for any reason it chooses. This is a viciously authoritarian State running entirely out of control. Of course they would prefer to keep every single thing in the world “secret” and away from public view — because they want and intend to do whatever the hell they wish, they don’t want you to know about any of it (until they do want you to know, at least one or two things, when you’ll be sorry you found out even that), and they don’t intend to ever have to account to anyone for a single damned thing.

I therefore repeat: dump every single goddamned document you can get your hands on in toto, and make every last bit of it available to the public. It’s the public that’s paying for all of it, and not merely monetarily — and it’s the public for whose benefit everyone claims to be working (another similarity between the journalists and the State). If it’s all for the public, doesn’t the public have not only the right to know all about it, but the first right to know about it? But, no: you’re too stupid to know what’s best for you. Your betters will have to make these determinations. They’ll decide what you can know, how much, and when.

Bullshit, all of it. These are the dishonest, insulting arguments of power used to justify itself. To hell with it.”

@ THD #86: I knew some of you would understand that. I’m still stuck back at the inflowing fiber optic cables being split in half, or ‘half light info’ splitting…or something. But they are grabbing the data *before* it gets to the company’s big servers, right? That’s what ‘upstream’ means?

I just thought of another analogy which may or may not help us understand what all of this means in a more general way. When the mortgage crisis first broke, we were all trying to wrap our heads around the neat little bypass of county and state monitoring of our properties called MERS – something cooked up by the mortage finesse guys to enable them to slice and dice mortgages into hamburger and sell them as securities. (Interesting, there’s that same word ‘security’.)

I know, this is apples and oranges, but it is a way to do large scale scooping and snooping and pretend legality – certainly there’s no way to hide the big black boxes that NSA operates, but they’d like to get it all prettied up, I feel sure, before the public starts sniffing around their own privacy issues, just as they did with MERS, and look what a mess that’s gotten us into, Stan and Ollie.

This is bigger by far than the mortgage crisis. And where that involved the whole financial system worldwide, this is our wonderful democratic and free government indiscriminately grabbing everything ‘just in case’. So, yes, we’re hot on the trail, and sunshine is, as Mr. Silber pointed out, the name of the game. I would say we are off to a good start, but no dillydallying, you hear?

Thanks, wendydavis, for tracking that down and giving worthy attention to it.

Optical splitting is the technical name for the process by which one data stream becomes two identical data streams. It is sufficient to know that information coming into a telephone central office (either an international point or a domestic point–technically the switches are different) becomes two identical streams, one going into the telephone central office equipment and the other going into whatever equipment and connections NSA has in their special room. Because the NSA data stream is coming directly from the source (the backbone line coming into the central office), it is “upstream” of the NSA storage system. No need to worry about the cables, just the data. It is before it gets into the telecom’s international switch or the switch that distributes it to customers and is the stage before it gets into NSA’s servers.

USians are so US-centric that they don’t realize that all countries with the technical capability to do it are snooping on the internet and have more or fewer secrets. The global public needs to know all of this and not just what the US is doing.

What I see Snowden and Greenwald doing is limiting their legal liability in a reasonable court by vetting the information. The more that becomes known, the more it undercuts the charges of treason. Bradley Manning did the same, and some day folks might realize it. It also puts establishment media outlets in civil disobedience and in some degree of legal jeopardy releasing what they have released. But for ordinary citizens, this restraint allows them to cut them slack over purist arguments that they were breaking the law. (Yes, indeedy they were breaking the law–bigtime–that’s what civil disobedience is about, breaking the law.) And in some respects the Guardian and WaPo are hostages that allow the stream of information not to be immediately shut down.

I suppose that Silber would prefer a major document drop that everyone who wanted to could pore over. It is unlikely that as many people would know the key information if it were released that way. How many people know what’s in the Wikileaks files as compared to know about NSA surveillance now?

But it is not just our government snooping on us, it is our government and complicit other governments snooping on us and on folks in other countries and hacking the systems of the yet other countries that are not complicit in the scheme.

The “outward facing”, i.e. spying on foreigners, is considered by USians as a legitimate mission. But the way this is being done is moving to a global surveillance state on everyone with personally identified electronics.

The MERS scandal was really about fraudulently signing off on conveyances, affidavits, notary fraud, but yes, all in the name of making bundles into derivatives in which no one could discover their value. The swaps were the most evil: betting on shit you sold to fail, lol. Some way to make a livin’, eh? Counties then never received any of the fees they were due, and folks in foreclosure never even got to make sure that the deed of conveyance was even genuine! The ‘demand the deed!’ movement was cool; dunno how many folks got to stay in their houses. The cases sure were backed up in court, as are the reverse-motgages now. So many possible acronym agencies involved…so little time. Arrrgh (my FiL just went through it all.)

But I think I see your comparison, and this one’s tougher to understand for me and many others. The polls are killin’ me, juliania, since I reckon not too many people even know what the pollsters questions might mean given the complexity of it. Are all of them push-polls designed to elicit reflexive responses? They seem all over the map for now.

Well, I hear Arthur, and I’d like it if we know what questions to ask. As kgb put it so well: ‘…annoying how many holes remain.’

Arthur: he’s been in the process of dying over the last year at least, and is broke, always asking us to send a buck or two his way as we can…and do. He’s been flipped out that no one paid attention to a lot of his recent posts (don’t know he knew that), nor took his advice on steps to take. Aprescoup once ventured the opinion that Arthur was an anarchist, given his blogroll. That might make sense of his wanting all the documents dumped. I’d figured that Snowden and Glenn were just taking it step by step to keep building on the story, and gaining global support for it all; but I sure dunno.

Thanks for causing me to go read the piece. And I’m sure we’re all looking forward to the next revelations. ;~)

One of the pieces on a perhaps 2003 article had one NSA official saying that they meant to own the internet in its entirety. A good step toward TIA globally.

Thanks for the #89, too; I may begin to see da light yet. ;~) But re: Arthur and the total dump, he seemed to think the slides being withheld might provide some of the largest clues. That reminds me of a detective’s quip on a pbs mystery the other night: ‘A clue is often just another name for a (criminal’s) mistake‘.

Yes, my point was a broad one, wendy, that the technology necessitates it be used and they don’t really look far enough ahead (or blackly maybe they do) to see the consequences of storing and manipulating data just because they can and what a sense of power to be able to do this and have it storagewise – control, control, control…laws are just as malleable as the technology – simply an exercise. So the mantra can be ‘It’s legal.’

It just made me think – okay, wall street has MERS, so hey, cool guys, here’s our acronym – PRISM. Way to go, Blackberry loving POTUS. It’s the technology, stupid. We are So With It. Even have a Power Point Presentation. Who needs a tie? – love those rolled up sleeves.

I am so ashamed of voting for this guy in 2008. But then, voting is so deja vu now, laughable really, quaint. They really used to do that – to vote?

Ah; shekissesfrogs brought the quote to us a few years ago. Great ones I stick in a blogging help file on my desktop. It’s so old that it’s labeled ‘TPM help file’, lol. Too many pages by now, no way to search it that I know of… ;~)

But yes to all you’ve written, arrow. And it’s a good thing that Snowden seems to have such strong core principles and resolve. The massive discrediting he’s receiving will put it to the test, which he knew going it, and good on him. If our goodwill could help him, it might be helping Brad Manning more, but still…who knows what thought and prayer can accomplish, not to mention the seeming futility of…blogging. ;~)

The government doesn’t have to copy Wall Street to generate acronyms; the US government invented the use of acronyms in cute ways. I’m trying to put the right words together to call the new global system that embraces all of the pieces parts JEDGARHOOVER. I’ve got the last three letters: vaccuming every record. The G just has to be “global”.

In other news, emptywheel and Kevin are covering Mueller’s testimony to Congress. Apparently he thinks that the “hoover up all the metadata” authority permits him to use that metadata in searches during criminal (i.e. day-to-day FBI) investigations. Which raises the question CTut keeps raising: how come the Fusion Centers aren’t tracking banksters?

I’m waiting for some PTB heads to pop up with a “Say whaaa?” expression when they realize that their data is stored and available to the Edward Snowden’s of the world. Or maybe that’s where the “treason” accusations are coming from. They have realized it.

Ah, I see it better now, j. Got too many things goin’ at once again to read well.

It just occurred to me about the global angle of all this: it must be possible that some of the information gleaned from all this could be used in highly political ways, much like the CIA/JSOC/Xe drones are used to outsource assassinations of political enemies, as in Yemen, who knows where else. And sure it could be used in aid of global extortion, bribery, all the tricks that Empire has come to use so cavalierly and brutally.

Meaning, if globalization has meant the erasure of sovereign boundaries,
except to control a nations citizens more ably, who knows what the real aims of all this is, and who is really at the helm. Or is it more of a free-for-all now that it’s been up and running for so many years already?

Love the great quip, and the link, but er…’synthetically produced genetic material can be patented’. Dunno anything about the subject, but that bit creeps me out. Is it like genetic manipulation by nanotechnology or something? Or inserting foreign dna like Monsanto, et.al. do? Like in human embryos? Ack, too many things to not know, lol.

The other thing to bear in mind is the likely nature of PRISM. It seems like it’s main purpose is essentially to process the paperwork and handle an interaction with the ISPs – looks like *after* all the sneaky stuff has happened and whatever passes for a warrant has been acquired.

IMO, the typical establishment-defenders have gone the PRISM, PRISM, PRISM route as a defense – and folks are following the lead. It is likely that (a) when the details are revealed, PRISM is basically office software and (b) as the default portal for requesting data from ISPs, any successful investigation that involved data from a participating ISP likely would have used PRISM to make the request.

It’s the one they *want* to talk about …. “ignore all those questions about how we GENERATE the list of names of Americans to spy on, just remember, when agents finally go to the part of the system that grabs all the data from your inbox, we totally used a FISA-compliant mechanism to make the demands!”

I think Marcy Wheeler and a few others are a bit out over their skis in some regards … especially in analysis of the Zizi thwarting (super in the weeds, just giving it for reference). They’ve got her befuddled between section 215(collection) and 207(official surveilence) … she’s still not getting that the part in the middle (big-data mining) isn’t regulated by *either*.

Merkley is starting to get it though ….

Merkley pointed out that the language of Section 215 of the Patriot Act did not govern database queries from collected phone records. “These are requirements to acquire the data” in the first place, Merkley told Alexander.

So you are saying that PRISM manages the data about the transactions between NSA and the data providers (Google, etc.) so as to provide a custody trail of the data that they are sucking out through their requests? Wonder if that creates tagging that provides a trail for sneaking stuff in under the color of law.

Meaning, if globalization has meant the erasure of sovereign boundaries, except to control a nations citizens more ably, who knows what the real aims of all this is, and who is really at the helm. Or is it more of a free-for-all now that it’s been up and running for so many years already?

Sorta puts the Trans-Pacific Partnership secrecy in a different light, eh.

Okay, now I’m back to bumfuzzled, and I shouldn’t have even tried to read Maarcy’s piece when I’m so jammed for time (big natural food coop order due, haven’t even started, for one). But juliania brought this Pilkington and friend piece that refuted the NSA ‘wins over terrorist’ piece.

But, crap; I’ll read yours a third or fourth time, see if it sinks in better; the terminology is all so new to me.

And did you bring your understanding to Marcy’s thread? Those of you who DO get this stuff need to (she sayed hopefully…). Thanks, kgb; hope you’re doing well. By the by, I loved ‘out over their skis’, what an image. ;~)

Corporations have no national loyalties. Their only loyalty is to their own present and future profits. Since they now control the American government, it’s no surprise that they are using governmental power to advance their own interests.

There’s a word for that kind of system. Mussolini coined it and defined it, and Franklin Roosevelt simplified the definition. It’s fascism, pure and simple. It can be defeated, but the price will be high. It always is.

The TPP could make all this shit look like child’s play once they take away your sovereignty in the name of global neo liberalism. Or— they may find a way to use it all against us. But that may be a little down the road. first make sure the profits are secure.

As if it needed a scarier light, lol. Need some hacks on *that* language (would it do any good?) Some days all ya can do is laugh. I looked earlier for that (ahem) rather retributive justice video to the Beach Boys’ ‘Wouldn’t it be nice?’ Just too much fun, but I can’t find it. So many villains, so little fun so far. ;~)

Guess we can always go to Boots Riley again, lol. (Ack; it has an ad now.)

No. I didn’t read the Ars article … and I’m not seeing it on the site (or at least identifying it).

In some ways it seems like six in one hand half dozen in another. A tag “ANI=8885551212″ it would largely be functionally equivalent to a database field “ANI” with the value “8885551212″. (tech note: ANI) as far as writing a query goes. The primary difference as a developer is that a defined field provides an enforced schema so you explicitly know the data-type, character encoding, etc.

I should probably refrain from comment until I read the article, but superficially I don’t see how one approach over the other would provide a specific benefit in terms of minimizing collateral data extraction related to innocents. That said, I can think of a bazillion reasons you’d want the analysis system to have an ability to flexibly tag structured data in an arbitrary fashion.

She’s riffing off the Mueller testimony this morning. And wondering/speculating that our focus should be broader than NSA. Who else has authority to vacuum data and what are the constraints on the way they use it as defined under section 215 of the PATRIOT Act?

Clearly FBI has the NSA storing the data they have authority to vacuum up. But for what legal purposes can FBI put that data? Can they use it for ordinary criminal investigations, for example? This gets into the “just how blatant an invasion of the 4th amendment has Congress created here” territory.

And which agencies beyond FBI have this sort of authority?

Down in the legal/administrative weeds on this one she is, not the technical weeds.

At the point an investigator decides they have probable cause to request information on a target there is a weighty task which is purely administrative. That task basically boils down to creating the specific documents that must be sent to each ISP based on a NSL or warrant.

PRISM presents an agent with a bunch of check-boxes where they choose which ISPs to target and what specific data they are requesting from each. Then they push “GO” and PRISM creates the required documentation and sends it off – one set for each ISP customized based on the surveilence features requested.

It essentially saves an agent from having to do two days worth of paperwork when they want to go after a target and … gives them a nice standardized report for distribution when all the data is returned (saving them another day of paperwork).

If I were designing the system there would also be a “Generic ISP Request” to auto-generate paperwork and provide a data-entry point to manually integrate information returned from the non-participant ISPs within a unified platform.

So it’s a tool for creating minimized requests of the major internet services–sort of an automated NSL.

And in response, the internet service bundles up an extract and drops it through some interface? From which NSA picks it up and loads it filling out the remainder for the report, which then gets distributed to the “customers”?

What if that general over at the NSA works for them? Wouldn’t that sort of change the calculation from their perspective a bit? He’s not even subjected to elections; consolidate enough power into his hands and you don’t have to worry about much of anything.

And you must admit … all those Occupy protests *were* kind of pesky for the folks we both agree run things. And ultimately … NYC and Oakland *did* pretty much conduct the prototype operations that Turkey has been emulating to deal with their own pesky little sit-in problem. And it sure does look like these NSA capabilities came into play in terms of responding to protest leaders … as if these corps are totally able to direct the machine’s focus anywhere they want.

There is a side of the Democratic party rearing it’s head right now every bit as scary as anything the GOP ever manifested … and they are being joined by the worst of the GOP old-guard. It feels kind of pointless debating labels that have long lost true meaning and are just tossed about as pejoratives. Whatever you want to call what’s going on right now … it’s headed in a scary-bad direction and rapidly getting worse.

And Sheldon Wolin calls it ‘inverted totalitarianism‘. Works for me for now, (even if we consider it’s no longer just the US involved, but their affinity groups and slaves). And as you say, it’s getting worse so quickly now.

Thank you for all this; I’m so glad some others can play at your level, and understand the knowledge you’re bringing.

Plus all the private security firms and their millions (?) of employees. That’s kinda what I meant by a ‘free for all’, and yes, once those pennies drop. Somewhere in the may be the whiff on the air taking hold among ‘them’ that this is an Empire in decline, and history seems to indicate that”s when Empires become the most dangerous and oppressive.

Yes, but he is subject to the political process and lets hope he can control all those folks, like Snowden. Not an ideal set up. Better to modify the security state. BTW he is now also on a hot seat and has said he wants to declassify some of that shit.

On the occupy thing do we know if NSA was involved or surmise? I ask bc they wouldn’t need it. Just the cops in NY with pepper spray and dogs. And the threat of jail when you may have a job to go to.

Something needs to be done. But until now most people did not know about this. I am hopeful some changes will be made. Even Rand Paul alleges he is upset. And we still can vote. And
one thing politicians do is pay attention to polls. If the FISA court were modified we might all be satisfied.

But I don’t want to minimize it all. Secrecy is not the best way to go and we need to stay,on it.

I realize that this comment wasn’t to me, but who is the ‘he’ that you hope ‘can control all those folks, like Snowden’, and why? And who is on the hot seat?

Yes, there was a full-throated collaborative effort nationwide to quash the Occupy movement, and many FOIA’ed documents to tell the story. FBI, DHS, dunno how many security acronyms. You can check with http://www.justiceonline.org, for instance. And documents showed that the Big Banks in NYC donated boatloads of money to the police dept. as a pretty clear quid pro quo for cleaning out Zucotti Park. Many more examples nation-wide, Portland, Oakland, Boston, for a few.

Modify the security state, or blast tons of sunlight on it” Can it be unwound and will we believe it if they tell us they are?

No, politicians rarely pay attention to polls; especially at this point, they don’t have to. You vote red or blue, and that’s voting?

This shit may be called ‘legal’ by Obomba’s twisted terms, as he claims that drone assassinations are ‘legal’ as authorized by the AUMF. At this point, repealing that might do some good, but even then, the Unitary Executive that Bush began, Obomba ramped up…will be very hard to reign in.

Of course he wants to declassify *some* of that shit. Just enough to get everyone off his back.

On the Occupy stuff we know the FBI used their anti-terror authorities in relation to those folks who were arrested before NATO Protests Chicago … Tarheel Dem could certainly speak better than I on that. There are also strong suggestions throughout data revealed in the HB Gary hack. Then there’s stuff along the lines of what Wendy highlights. Yeah, it’s pretty evident mission creep has set in in a big way.

I agree that hopefully this results in positive movement. The idea another Snowden might up and wander off the reservation worries me way less than the idea someone like Alexander has accumulated so much concentrated power with so little oversight.

I’m not sure which political process you are referring to. There are multiple ones at play. There is the political process of responding to public concern. There is the political process by which corporations that are concerned about being tracked and corporations that make big bucks supplying the products and services for tracking divide up the Congress–note the stances of Barbara Mikulski (Sen-Ft. Meade) and Jon Tester (Sen-whatever MT interests have him). Then there is the political process by which the President ensures that the military and intelligence communities trust him enough to follow his orders. And the political process of advantage and disadvantage within the Village that has impact on some of the other political processes.

The White House definitely is, until Syria steals the headlines, on the hot seat. Their best move would be to allow an independent investigation with a long enough scope to include both Republican and Democratic (or Bush and Obama) administrations. That would allow the fig leaf of “at least I brought it under the color of law”. So far, it doesn’t look like that is the direction of the White House. Their surrogates are still trying to stuff the story back into the box.

Was NSA involved in the suppression of Occupy Wall Street? Gee, that is something I would like to see in an honest investigation. Here’s what we know. Very early on, the DHS Federal Protective Services (the building security for federal buildings) were patrolling along with local police where there were federal buildings nearby. Either the NYPD or the CPD or DHS used a DHS/DOJ funded nonprofit research and training agency for police executives as a forum for local police departments and DHS/FBI to discuss strategy. DHS Fusion Centers were involved in the realtime coordination of police actions relative to Occupy locations and actions.

What is not known is where the accusations of potential terrorism came from. Typically local police just make them up. But there is the possibility that some analyst at NSA made a big data correlation and came up with a narrative plausible to DHS/FBI and the police of a “possible threat”. You know, “We are 51% sure that there is the 30% chance that someone with an inclination toward violence is 20% possibly in the leadership of the Occupy group at xxxx.” But the police, always in worst case possible mode or in SWAT riot gear theater, took that as their marching orders. It would be very interesting for someone independent to have the subpoena powers to get the emails of what went on. My hypothesis is that it was convenient story and NSA had zero/zip/nada information that would support it.

The indictments in Chicago indicated that investigation had been going on since October 1. There was no reason given for the beginning of the investigation, but October 1 was the day of the first Brooklyn Bridge march that had a huge turnout and 700 arrested.

If you’ve been following the fallout from last week’s NSA surveillance revelations, you may have seen repeated reference to a certain “recent MIT study.” “Unique in the Crowd: The Privacy Bounds of Human Mobility,” published in Nature’s Scientific Reports last year, has been cited by multiple media sources… as evidence for why …. your metadata matters. Indeed, re-examined in light of the current headlines, the concerns raised by the study seem quite prescient.

OK I spent a little time with the MIT paper. One key conclusion is from figure 4 caption: It is easier to attack a dataset that is coarse on one dimension and fine along another than a medium-grained dataset along both dimensions. The study shows that it is fairly straightforward to randomly select unique traces (consecutive locations) even if the geographic resolution is low as long as the time resolution is high – and vice versa.

I’m glad I said individuate and not identify above. The point is that even totally anonymized data can be readily resolved into unique activity patterns that can then be matched to known patterns of individuals with much less effort. So, theoretically, it ain’t that hard to put names to anonymized cell phone data. Among other things, I think that this analysis shows that the use of throwaway phones to create anonymity may be readily defeatable — if they are frequently used (the data in the study was selected from heavy cell phone users == >1 call/hr).

I doubt that the totality of the data collected by NSA requires working this hard: I think they get the metadata to make it easier to sort out the firehose stream of the Narus backbone taps (internet+cell phone calls).

Ha; only came by on another errand, LeMoyne. Gotta read in the mornin’, I’m bushed. It er…looks like all that’s way smarter than I am, but I’ll try. But damn, I’m glad you came up with the words to fit THD’s acronym. ;~)

Welcome to FDL

Sign in with Facebook or Google+

OR use your MyFDL username

Toolbox

MyFDL is Firedoglake's community site. Anyone can participate by commenting on posts or joining groups to find other people in your area. Content posted to MyFDL is the opinion of the author alone, and should not be attributed to Firedoglake.