from the getting-more-interesting dept

So, the big story yesterday was clearly the report that Yahoo had secretly agreed to scan all email accounts for a certain character string as sent to them by the NSA (or possibly the FBI). There has been lots of parsing of the Reuters report (and every little word can make a difference), but there are still lots of really big questions about what is actually going on. One big one, of course, is whether or not other tech companies received and/or complied with similar demands. So it seems worth nothing that they've basically all issued pretty direct and strenuous denials to doing anything like what Yahoo has been accused of doing.

Twitter initially gave a "federal law prohibits us from answering your question" answer -- and a reference to Twitter's well documented lawsuit against the US government over its desire to reveal more details about government requests for info. However, it later clarified that it too was not doing what Yahoo was doing and had never received such a request. Microsoft's response was interesting in that it says it's not doing what Yahoo is, but refused to say if it had ever received a demand to do so. Google said it had never received such a request and would refuse to comply if it had. Facebook has also denied receiving such a request, and, like Google, says it would fight against complying. This still leaves lots of unanswered questions about why Yahoo gave in. Again, historically, Yahoo had been known to fight against these kinds of requests, which makes you wonder what exactly was going on here.

Former GCHQ infosecurity guy Matt Tait has one of the more more interesting threads about this news, arguing (in some ways) that it's both less and more than everyone is making it out to be. His basic argument is that this is an expansion of the PRISM program to include "about" targets. This has been discussed in the past, but under PRISM, the NSA could give tech companies "selectors" in the form of specific addresses and the companies were compelled to hand over emails "to" or "from" them -- but according to the PCLOB's report on the Section 702 program it did not include anyone emailing "about" the selector. Upstream collections (i.e., tapping the backbones from folks like AT&T) did include "about" selectors (and this information also flowed into other areas, enabling so called backdoor searches. And, as I speculated yesterday, Tait says that this latest news appears to be Yahoo now agreeing to use "about" selectors on its emails, which means that it's still part of PRISM, with a massive expansion.

Tait then notes that if James Clapper wants to clear this up, he should state publicly whether or not "about" collection is a part of PRISM. And if that's the case, he should also explain when and why PRISM was expanded to include this. But, of course, Clapper and the Intelligence Community tend not to want to explain very much of anything, leaving lots of people in the dark.

And, frankly, that's stupid. The Intelligence Community thinks that this keeps "bad guys" on edge, not knowing what's safe and what's not. But that's dumb. They mostly know to use more encrypted/secret means of communication when they need to. Instead, what you end up with is keeping the public on edge and not trusting services. I can almost guarantee that one of the early comments on this post will be some of you insisting that all the companies denying doing this are flat out lying. I don't agree with that, because the companies don't have a history of outright lying on things like this, but the way the NSA and other parts of the US government have repeatedly tried to pressure them and gag them, it's much tougher to take anything at face value any more. And that's not good for anyone.

Re:

It also leaves unanswered the question of what will they do when all the other countries in the world come calling. It will be much harder for them to refuse now that it is known that they have done it once.

Re: Re: Prove it

Just what I was thinking of. Microsoft was already shown (thanks to Snowden) to have given the NSA unrestricted pre-encryption access to all Hotmail, Outlook.com and Skype communications (probably without a secret order, since they're "friends").

So that's Microsoft and Yahoo! so far, it really only leaves Google with the much bigger cache of communications - obviously the U.S. government wasn't going to leave that honeypot just sitting there. What secret orders has Google had to follow so far?

Re: Re: Re: Prove it

"it really only leaves Google with the much bigger cache of communications - obviously the U.S. government wasn't going to leave that honeypot just sitting there. What secret orders has Google had to follow so far?"

Google *already* scans all gmail, so all Google has to provide is a search interface.

So Google can deny with a straight face, while Eric Schmidt becomes the next Secretary of Defense (i.e., de facto heead of the NSA).

Re: Re: Re: Prove it

The thing with Google is that it employs quite a large cadre of Kernel and other free software hackers, and they are unlikely to stay silent if they find evidence of NSA or other agency access without a specific warrant.

I can almost guarantee that one of the early comments on this post will be some of you insisting that all the companies denying doing this are flat out lying. I don't agree with that...Back in the early 2000s, there was a staggering report released which showed the NSA and FBI had access to the internet in ways people couldn't imagine. This was the "first" the public heard about the snooping.

And just like this article does with the statement above, people instantly ignored it because they didn't believe it.

Fast forward nearly two fucking decades when a person walks out with powerpoint presentations that the world finally believed.

Here's the thing: Has anyone ever questioned how the original report in 2000 came to be?

At the time, the world's operating system was Windows.

Perhaps ask Microsoft how the information from the NSA was leaked.

As I said many times, what's the point in trying to address these issues when the very first thing people do is say "No way. A company wouldn't do that."

Re: Re:

Re: Re: Re:

Well, they are not like Lavabit. They can't just close shop because they'd have to screw over their customers otherwise: they'd be liable to their shareholders and employees. I mean, most of those companies would have to close shop if they were forced to stop screwing over their customers anyway. So why throw away everything you have because the government wants you to do a bit more of what you are doing anyway?

James Clapper statement??

How would James Clapper issuing a statement clear anything up? He perjured himself to congress. When confronted he said he gave the "least untrue" answer that he could. He committed a felony and was never charged and he kept his job. No one will ever believe another word out of his mouth. In fact because of him every denial and explanation from any of the three letter agencies will be called into question.

Re: Re: Re: James Clapper statement??

We already know what we'll get with a Clinton back in office! No thanks!!! I'm not a big fan of Trump either. He's not a Republican. Just another big RINO. At least he's run things unlike Obama. Your husband being president doesn't qualify YOU to be president.

Hillary is just a big fat criminal liar. Trump is clearly no politician and says whatever is on the top of his head. There hasn't been a good Republican option in YEARS. It's been RINO's and the country has being going more and more left.

Re: Re: Re: Re: James Clapper statement??

Yes, Hillary is a criminal liar. Trump is a liar and a complete fraud. I guess the only thing Hilary has over Trump is she doesn't sound like an insane nut job off his meds. Hilary belongs in prison, not the white house. Trump belongs in a padded room and heavily sedated. Maybe I should start checking into countries to emigrate to unless one of them drops out and someone qualified gets elected. Unless that happens this country is going straight down the shitter.

Re: Re: Re: Re: Re: James Clapper statement??

Re: Re: Re: Re: Re: Re: James Clapper statement??

Both sides have a small minority of staunch supporters but for most voters I think it will come down to who you hate the least. They are both unqualified frauds. Whoever wins, brace for years of scandals that will make Watergate and slick Willie's BJs pale in comparison.

Re: Re: Re: James Clapper statement??

A lot of this began under Clinton (Democrat), Was greatly expanded under Bush (Republican) and Obama (Democrat) let it go on and even tried to defend it for a while after the Snowden leaks. Obama has a special hard on for whistleblowers. Now tell me it matters who is elected.

Re: Re: Re: Re: Re: Re: James Clapper statement??

Sorry, but the Dems are anything but near the center. THey have taken over education. They are taking over healthcare. They are looking at childcare now. The produce tons and tons of regulation. Soon they will have control over nearly every aspect of your life and before you know it you have a totalitarian regime. Time for the frog to jump out of the pot.

Re: Re: Re: Re: Re: Re: Re: James Clapper statement??

They are taking over healthcare.

If by "taking over healthcare" you mean "passed a requirement that every person in the country become a consumer of private health insurance or pay a fine, as originally proposed by the Heritage Foundation and previously supported by Republican Party leaders including Newt Gingrich, Bob Dole, and Mitt Romney," then yes, the Democrats definitely did that.

Re: Re: Re: Re: Re: Re: Re: Re: Re: James Clapper statement??

Of course. There's a lot more common ground between the Tea Party and the Occupy movement than either side is willing to admit, and it's in the major parties' and their donors' best interests to emphasize the differences rather than the similarities.

I think my analysis of the ACA is on point: it was a Republican idea until the Democrats started supporting it, at which point Republicans immediately declared it to be socialism and refused to support it. It's not about the content of the law (which, for the record, I believe is deeply flawed but superior to the system we had before), it's about a two-party system defining itself in terms of "we stand for what they don't stand for."

It was a compromise bill. It should have meant compromise. But the only side that was compromising was the Democratic side. That's not how compromise works.

But we're pretty far off-topic at this point. Unfortunately, both major parties largely favor the type of surveillance the article is talking about.

I don't agree with that, because the companies don't have a history of outright lying on things like this, but the way the NSA and other parts of the US government have repeatedly tried to pressure them and gag them, it's much tougher to take anything at face value any more. And that's not good for anyone.

I said it yesterday and people much smarter than me have been pointing this since Snowden. The best comment yesterday was something like: assume everything is compromised and act accordingly. And I'm already doing it by encrypting whatever I find sensitive but can't remain in an offline storage for some reason.

Ironically this may push towards these services using open source, end-to-end encryption to have a good marketing point. So we may actually emerge in a better state after all this surveillance is scaled back (hoping it will).

One legal, easy way to protect customers' cloud data would be to serve the data, RAID-like, from multiple countries. In a RAID-2 system of three or more drives, bits are stored sequentially across all the drives save the final one. The final drive merely records a bit that indicates whether the sum of the other bits is even or odd, failure-proofing the other drives.With RAID drives located in multiple jurisdictions, subpoenaing one country would only recover info of a single RAID drive, useless jibberish.

Re:

"One legal, easy way to protect customers' cloud data would be to serve the data, RAID-like, from multiple countries. In a RAID-2 system of three or more drives, bits are stored sequentially across all the drives save the final one. The final drive merely records a bit that indicates whether the sum of the other bits is even or odd, failure-proofing the other drives.With RAID drives located in multiple jurisdictions, subpoenaing one country would only recover info of a single RAID drive, useless jibberish."

Good, but not good enough, due to "3rd party doctrine".

You now have to "stripe" across multiple vendors -- e.g. Box, Dropbox, etc.

Re: Re:

Re: Re: Re:

I suppose it depends on how the key exchange is handled. If your data is encrypted end-to-end, and transmitted through a separate source from your encryption keys, then that should mitigate the problem of MITM attacks etc.

They are just not doing it for the government....

When Google says that they have never and would never build such a system for the government they aren't strictly speaking lying.

They wouldn't have had to as they already have one. What do you think scans all of your GMail as part of their advertising operations?

Now I'm not saying that Google has been re-purposing their exiting software to serve the NSA or other LEO's, but it wouldn't be the first time government actors piggybacked on existing advertising infrastructure. Some of the documents released by Snowden outlined the NSA doing just that.

Perhaps Yahoo just found a way to get the government to pay for building the software to let them do with their email what Google's been doing with GMail all along.

Re: They are just not doing it for the government....

Except, passively scannig email and assigning ads to it, while similar, would require different software from the type yahoo is described as using. Funny thing, software can only do what its designed to do, and Google's ad matching algorithim likely doesn't include include the kind of frontend needed to produce emails for the government based on keyword selection. While yes, the could modify the software to do it, it would require google to build such a system for that purpose. Google's adwords software doesn't require it, so the build would be for the government.

Mike Masnick

Your head is in the sand! I sometimes wonder if you should be reporting on technology because you have some willful blind spots regarding a few things.

In my opinion, given the things I have already seen... there is just no way to square away the following comment with sanity!

I can almost guarantee that one of the early comments on this post will be some of you insisting that all the companies denying doing this are flat out lying. I don't agree with that, because the companies don't have a history of outright lying on things like this, but the way the NSA and other parts of the US government have repeatedly tried to pressure them and gag them, it's much tougher to take anything at face value any more. And that's not good for anyone.

Not ONLY do these companies have a history just outright lying, they have a history of outright lying ON THESE THINGS!

Speaking untruthfully without lying

These are big companies. I think it entirely possible that the company could have some employees who are knowingly complying with this type of thing, and yet issue a denial that the spokesperson issuing it believes to be true. Yahoo itself provides an example of this. Per the article, the security group initially thought that they had found malware left by an intruder. It was only later that they discovered that colleagues from another division in the company had installed that malware, under orders and approval from the top. Given that, it seems very plausible that the spokespeople who issue these denials could be unaware of what was done behind closed doors in another division, especially since, almost by definition, the malware division is intentionally secretive. There is no monthly meeting where the company tells everyone what every division is doing at a detail level sufficient for this type of misconduct to come to light.

It's why I implemented by own email servers.

It isn't hard. It didn't take more than a day. There's a pretty good guide. Once it is up and running it is pretty much service free. It is no harder to do updates than it is to do them on a computer. Try windows 7 updates these days. Can take days to update. A simple command in Linux set up as your email server and you can update. Using SSH you can even do it remotely.

If this revelation bothers you give it a try. Don't get bogged down in the imaginary barriers professed by others.

Most guides cover spam, security, malware scanning, etc., so you aren't left hanging out there wondering.

Re: It's why I implemented by own email servers.

So you're concerned that you can't use your ISP for E-Mail because it might let the government monitor your inbox, and you think the solution to this problem is to set up a home server that sends and receives E-Mail *through that same ISP*?

Re: Re: It's why I implemented by own email servers.

Re: Re: Re: It's why I implemented by own email servers.

SMTP/STARTTLS doesn't prevent your ISP (or any other relay between you and the recipient) from intercepting the content of your E-Mail in transit.

It's true that "if he configured it with proper encryption the ISP isn't a concern" -- but in this instance "proper encryption" means a client-side solution like PGP. In which case it's irrelevant whether he's using his own server, his ISP's, Yahoo's, or anybody else's.

Re: Re: Re: Re: Re: It's why I implemented by own email servers.

I was going for brevity. If I were to go into all the reasons running your own private mail server for security is a dumb idea, we'd be here all day. But here's a Techdirt article on the subject from August:

Yeah, of course... and they're lying through their teeth.

"hur dur - big companies deny wrongdoing"

Of course they do, Mike, and they're absolutely lying through their teeth when they do so. They've lied about it in the past, and they're lying about it now (especially Google)... So the question is not "why did yahoo give in", it's "why did they all give in and lie through their teeth later (including Google)". And secondly, "why do fan-boys of said companies go out of their way to believe the false denials (including those of Google)?"

Re: Yeah, of course... and they're lying through their teeth.

To be clear, all the companies mentioned in the PRISM (who are many of the same companies) denied it then too.

And as Christopher Soghoian of the ACLU said in response to that, either the companies are lying through their teeth OR the government has cracked into their server farms. That is if you believe the PRISM leak, like the author of this article does.

Re: Re: Yeah, of course... and they're lying through their teeth.

To be clear, all the companies mentioned in the PRISM (who are many of the same companies) denied it then too.

No. This is wrong. They denied what the initial Guardian & WaPo reports said -- that PRISM gave the NSA unfettered access to their backend systems. That turned out to be WRONG. The tech companies were correct and the original reporting was incorrect.

To be clear I don't trust anything (at least when it comes to computers) that I can't verify for myself. Privacy is too important for anything less than paranoia. I can't verify what code Yahoo, et al are running on their computers so I don't trust what they say about it. What I would trust is if Yahoo let native clients encrypt messages in a way (say using DIME) that they couldn't do this scanning.

All I really know about the Snowdon leaks is that they are far too possible.

That said today we sometimes have to trust a company's assertions, but it's my goal in life to get away from that. Plus I've found prettier software this way, and the only inconvenience I'm facing is telling people I'm not on Facebook.

To be clear I don't trust anything (at least when it comes to computers) that I can't verify for myself.

But as Ken Thompson demonstrated, such verification is never truly possible; unless you not only audit the source of every program you use but actually write the bootstrap compiler yourself, at some level in the stack you have to trust somebody else when they assure you that there's no malware being injected into the program at compile time.

(For this we have the wisdom of crowds; if GCC, LLVM, et al were injecting malware at compile time, somebody would have noticed by now.)

Paranoia is a good default mode to be in. You should naturally assume that every website you go to is logging everything you do, and every E-Mail you send is accessible to malicious actors including governments. It's good to push back on this stuff, and to take precautions where appropriate (VPN's if you want to conceal the source of traffic, PGP if you want to send E-Mail that can't be observed by a third party, etc.). But somewhere in the chain you have to trust somebody other than yourself.

blow back

Call me gullible, but I think that the blow back from the Snowden leaks have dissuaded most tech companies from willingly going along with these kinds of measures. Sure, they will ultimately comply with a national security letter, but not without first making a legal attempt to fight it.

Yahoo's poor finances might have motivated them to acquiesce. Facebook and Google don't have such burdens.