Posted
by
timothy
on Saturday September 19, 2015 @06:07PM
from the tough-day-indeed dept.

New submitter jack_babylon writes: On September 14th, Symantec's subsidiary certificate authority Thawte accidentally released a "small number" of " "inappropriately issued" security certificates, apparently intended for internal testing only. However, the fact that these were logged in the wild by Google (and, apparently, DigiCert) seems to indicate that they escaped the lab, at least far enough for a false google.com cert to raise the appropriate red flags. This sounds similar to the recent acts of poor judgement that got CNNIC's certs removed entirely from Firefox and Chrome, if more limited in scope and more quickly addressed (through, among other things, termination of some Symantec employees). (And like all reports one hopes go away quietly, these were released in the dead of a Friday night — h/t BoingBoing for noting this news.)

Let's see. Based on what information we have so far, which almost certainly isn't the whole story, the incident happened on Friday night. It's now early Sunday morning in the US and some employees have already been terminated, presumably for gross misconduct since mistakes can (and do) happen, so that alone implies this was probably a willful act and the perpetrators were somehow either caught in the act or there was a clear audit trail when the fake "google.com" certificate came to light. There have already been allegations that the US' TLA agencies have been planting employees in US tech companies for such purposes so OP's conclusion isn't completely out of the field, although it could just as easily have been a large criminal organization or foreign government. Due to the requirements of making effective use of fraudulent certificate it's highly unlikely to have been a get rich quick scheme dreamed up by those involved without some form of government/organized crime support.

I expect this will blow over very quickly for Thawte. They appear to have procedures in place to tie specific certs to specific individuals, will no doubt already have revoked the certificates concerned, and we can probably expect some explanatory notice to be published in the next few days to explain their version of events; there really isn't much more they could have do in the face of rogue employee. They should also be handing what evidence they have over to law enforcement for potential prosecutions, which could get interesting if the individuals involved were indeed working at the behest of a US security agency...

You do understand that creating certificates for other companies is the whole point of a certificate authority like Thawte, right? As an example, Google's certificate on www.google.com was issued by Google's own CA, and Google's CA's cert was issued by a CA called GeoTrust (which is also owned by Symantec, like Thawte). Trust in Google's identity is established because your browser has been told that GeoTrust is a trustworthy entity.

The Certificate Authorities (Thwate, DigiCert, etc.) sign certificates. Anyone can generate a Certificate Signing Request for any set of names. It is up to the Certificate Authority to verify that the request was generated by an authorized representative of the named entity.

Of course a CA employee can generate a "bogus" CSR. Anyone can.

All this shows is that Thwate's controls around ensuring that proper validation is completed was defeatable by an insider. It doesn't highlight any new vulnerabilities in the

The ideal situation is that the Certificate owner generates a signing request and has that signed, so the original key does not go outside the certificate owner.

However, there is nothing in the current setup to prevent a certificate authority from generating a request in the name of any domain and signing it. That's what appears to have happened here.

The real question is 'why?'. The explanation ("testing") doesn't pass muster. Someone would have to deploy these certificates on a service that was either a Google property or was masquerading for a Google property. Does Google outsource the deployment of certificates? I would doubt this very much, which suggests that this wasn't so much an accident as the influence of a TLA.

As for browsers, I should be able to remove Thawte from the trusted chain

Go ahead. In Firefox, hamburger->options->advanced->certificates->view certificates. Find the two headings for Thawte and set all of their entries to "distrust". I've no idea exactly how much of the web will stop working correctly after that, but it's not hard to do.

I should be able to configure a warning if a domain has changed its certificate authority chain since the last time we saw it.

You should, and I'm sure there's some kind of add-on or setting for that, but I don't know what it would be off the top of my head.

Certificate Patrol would be that plugin : https://addons.mozilla.org/addon/certificate-patrol/But I cannot understand why it is used so rarely. There also used to be DANE Patrol which used to do the same thing while being able to handle multiple certs for 1 domain (think Google) made by Czech NIC, but it is not developed anymore and they recommend not to use it.

This is true. I know a Thawte partner in Spain that by default and without asking, generates a key and makes that key be signed by Thawte, then sends you he key and certificate by email. I was served this way even for a renewal!!

When I told the clerk I wasn't putting THAT private key on any server whatsoever and asked WTF were they doing, he told me it was std procedure. I just asked contact with a supervisor who understood something about security.

Any trusted certificate authority can issue certificates for ANY domain. This is the trust aspect that is required in a PKI.

Your browser gets a list of trusted root certificates and will accept any valid certificate issued by these CAs. On my windows 8 box there are 53. Any of these providers could issue certificates for any number of domains.

The failure here is that Thawte allowed those certificates to be issued for ANY reason.

Google is their own certificate authority and likely has no need for a relationship with Thawte.

Your browser gets a list of trusted root certificates and will accept any valid certificate issued by these CAs. On my windows 8 box there are 53. Any of these providers could issue certificates for any number of domains.

Worse those providers can issue "intermediate certificates" which also have the power to issue certificates for any number of domains. They can and do issue those intermediate certificates to third parties. So the list of root certs in your browser is not a complete list of entities who can issue certs your browser will trust.

There was recently an extension added to allow intermediate certs to be limited to certain ranges of names but that only helps in clients new enough to recognise the extension.

There was also recently an extension added for "key pinning" which makes bogus certs less useful.

Google is their own certificate authority.

At least when I go to google and check the cert I get a cert that has a google intermediate and a geotrust root. I don't see any evidence of name constraints on said intermediate cert though:(

Is this the first time that individuals were held responsible for online negligence? What happens to a CEO or CIO when data on millions of people slips out due to negligence? Has anyone ever been fired before (not just a flunky, but a responsible executive)?

The penalties for corporate irresponsibility are so small that there is no incentive to do the right thing. Actually, this case may be an exception because both Thawte and Symantec have a reputation to protect- they might actually fire an executive. The question remains (and you can ask the same of Wall Street criminals)- when has any executive ever paid for this kind of negligence?

Sure it's not a perfect fix but publishing the signatures of your ssl certs in DNS would care care of a lot of this low hanging fruit. A standard for cosigning your certs and pinning that cert would also help.

The end effect is needing to break multiple vectors not any of a multitude of root level CA's.

It's not that Google or Thawte have failed to correctly revoke certificates: it's that far too many people, at far too many sites and with far too many technologies, do not actually keep their signature authorities up-to-date. Because these people don't update signature authorities, they are unable to verify numerous valid certificates. These people then simply set their automated procedures, or make it their personal practice, to accept invalid certificates.

The notable case of this I saw recently is RHEL 5, where the signature authority information in the/etc/pki files managed inside the openssl packages, and updating openssl on a live server is likely to cause fascinating problems for an old, stable, production server. RHEL 6 sensibly put the root SSL keys in a separate package, but it's certainly not an unusual problem.

The problem is that any of the many entities your browser trusts can create a valid certificate for any domain and the browser will just accept it.

What we need is to move away from CAs and adopt a new system for storing the information needed to make a web connection secure. Storing keys in DNS and using DNSSEC to secure that is one option. And there are others (although I can't actually remember any of them off the top of my head).

If you have a situation where its impossible for anyone other than the actual owner of the domain to store a key, its not possible for a rogue CA (or a hacked CA ala DigiNotar or one that has been co-oped by a government or intelligence agency) to issue a bogus certificate or a bogus public key.

What I dont get is why there is no real interest from the people who came up with these alternatives to push them particularly hard and why there is basically zero interest from the people and entities who write the software that the web runs on (browsers, servers etc) to make any moves towards using these new systems.

Two other strategies are certificate pinning and certificate transparency. For pinning, you declare that only a certain intermediary CA (or root CA) may sign certs for your domain. So Google basically declares that all *.google.com certs must be signed by the specified Google CA. This information can either be hardcoded in the browser (for major sites) or relayed the first time the browser contacts the domain. So with hardcoded pinning, only Google can sign their certs. With http-pinning, another C

And the CA's have proven to be more interested in the sales part of the business than in the security part of the business. Thawte requires more checks than most systems, but the middleman certificate authorities such as DigiNotar have proven incompetent and apparently had their _signing_ keys stolen. And for many signature authorities, it's quite simple to request, pay for, and be issued a fraudulent new corporate SSL certificate for another company due to poor verification of the client identity. That's

There's nothing to find in the cert. The first method on pinning is in the browser itself. Microsoft can tell their browser which keys are allowed to sign for update.windows.com before they ship the browser.

That is the main issue here. As human beings, we understand there are limits to what we can do before we face really serious consequences. I mean jail time, not monetary punishment. Money is simply an expense. It might hurt, even hurt a lot, but it is not on the same level as being locked up.

Where is the jail time equivalent for corporations, and why do we continue to believe that we can somehow control them without it? To take back control of our worlds from corporations running amok over it, we need this.

To fire a number of employees means something very seriously went wrong. It also means the corporation allowed it to go wrong. This could be rogue employees the way someone robbing a bank with your car had lied to you when borrowing it, saying he needs to visit his ill grandmother urgently. Or it could be that you gave a gun to an obviously unstable kid when he said he's going to school and he's angry. You really should have at least asked a few more questions before handing over the firearm.

My brother had to cancel his honeymoon last month. He had his vacation time denied by Microsoft. The thing that has made him so angry is that since then several Indian coworkers have been allowed two week or longer vacations.

Microsoft has always had racist vacation policies. When I was there in 1982 the white employees weren't allowed a single day off while the few Indian guys we had were allowed to take two to three weeks off to go home to India. This racism is nothing new for Microsoft.

Racist is the correct word. When HR won't standup for you because you're white and admits freely that the company's policy is to not allow vacation time to whites, but allow three weeks contiguous every year to the Indian employees, then you are past the point of no return.

You just described HR everywhere. They protect the company and not the employee. After working for almost a dozen different startups in the Seattle area, I'm fedup. I haven't had an entire week off since 1994, but most of my Indian coworkers get two or more weeks off every year. Of course complaining to HR does no good.