The heartbleed bug is bad news for internet security

Many websites rely on SSL / TLS to encrypt communication: everything from passwords to credit card numbers to emails. OpenSSL is a very widely used implementation of these encryption protocols.

Right now, the internet is abuzz with the news of the ‘hearbleed’ bug. Because of a flaw in OpenSSL, attackers can extract 64 kilobytes of information from a webserver for each ‘heartbeat’. This information can include secret encryption keys, usernames and passwords, and other kinds of sensitive data.

EFF has long advocated for websites to support HTTPS instead of plain HTTP to encrypt and authenticate data transmitted on the Internet. However, we learned yesterday of a catastrophic bug, nicknamed “Heartbleed,” that has critically threatened the security of some HTTPS sites since 2011. By some estimates, Heartbleed affects 2 out of 3 web servers on the Internet.

Heartbleed isn’t a bug in the design of HTTPS itself but rather the result of a simple programming error in a widely-used piece of software called OpenSSL. It allows an attacker who connects to an HTTPS server running a vulnerable version of OpenSSL to access up to 64KB of private memory space. Doing the attack once can easily cause the server to leak cookies, emails, and passwords. Doing the attack repeatedly in a clever way can potentially leak entire encryption keys, such as the private SSL keys used to protect HTTPS traffic. If an attacker has access to a website’s private SSL key, they can run a fake version of the website and/or steal any information that users send, including passwords, private messages, and credit card numbers. Neither users nor website owners can detect this attack as it happens.

We can confidently say that our shared servers, VPS guests, and dedicated machines are NOT vulnerable to this issue because they run Debian “Lenny” and/or “Squeeze”. The most common version of OpenSSL on our network is 0.9.8o-4squeeze14, and the “HeartBleed” vulnerability in OpenSSL’s heartbeat module exists in versions 1.0.1 and 1.0.2-beta.

The thing that has folks like Schneier most worried is the idea that a server might give up its private encryption keys to this attack. That would give attackers who had been logging encrypted traffic sent to and from the server a way of reading that encrypted data. Right now, there’s some preliminary evidence that this may not be possible, but the jury is still out. “It’s early days yet with the vulnerability, so precisely how well people can weaponize it remains to be seen,” Morgan Marquis-Boire, a researcher at the Citizen Lab, University of Toronto who also works as a security engineer at Google.

…

Yahoo’s Tumblr is already saying it. “This might be a good day to call in sick and take some time to change your passwords everywhere—especially your high-security services like email, file storage, and banking, which may have been compromised by this bug,” the company said in a post. SalesForce’s Heroku division is advising password resets too.

Make no mistake, this bug is BAD. It’s sort of a perfect storm: the bug is in a library used to encrypt sensitive data (OpenSSL), and it allows attackers a peak into a server’s memory, potentially revealing that same sensitive data in the clear.

Initially, it was reported that private keys could be disclosed via this bug, basically allowing attackers to decrypt captured SSL sessions. But as more people start looking at different sites, other issues have been revealed – servers are leaking information ranging from user sessions (https://www.mattslifebytes.com/?p=533) to encrypted search queries (duckduckgo) and passwords (https://twitter.com/markloman/status/453502888447586304). The type of information accessible to an attacker is entirely a function of what happens to be in the target server’s memory at the time the attacker sends the request.

You’re protected from the Heartbleed vulnerability because you have CloudFlare turned on for your website. We fixed the flaw on March 31 for all CloudFlare customers, a week before it was publicly announced.

Heartbleed (CVE-2014-0160, http://www.openssl.org/) is a flaw in OpenSSL, encryption software used by the vast majority of websites to protect sensitive information. This vulnerability in OpenSSL allows an attacker to reveal up to 64KB of memory to a connected client or server. This flaw could expose sensitive data such as passwords or usernames – even when you thought it was encrypted.

NO IMPACT ON CLOUDFLARE SERVICE. Our team has conducted a comprehensive security review to ensure our customers were not impacted. One concern is that an attacker had access to the exploit before March 31 since the flaw was present since December 2011. We’ve seen no evidence of this, but we’re proceeding as if it is a possibility.

PRIVATE KEY DATA. Our security and cryptographic team has been testing the possibility that private SSL key data may have been retrieved. We have been unable to replicate a situation where private SSL key data would leak. We have set up a challenge to see if others can exploit the bug. See more information on our blog:

NEW CERTIFICATES FOR EVERYONE. Even though we have not been able to use the exploit to leak public key data, we’re proceeding out of an abundance of caution. We’ve begun the process of reissuing and revoking the keys CloudFlare manages on behalf of our customers. To ensure that we don’t overburden the certificate authority resources given the scale at which CloudFlare operates, we are staging this process. We expect that it will be complete by early next week.

GENERAL RECOMMENDATIONS FOR SAFE WEB HYGIENE
There are some precautions you can take to protect yourself from the Heartbleed bug.

1. GET CUSTOM CERTIFICATES ISSUED. If you’re using CloudFlare custom certificates, have your certificate authority reissue you a new certificate. After it is installed and confirmed working, revoke all previous certificates.

2. UPGRADE OPENSSL ON YOUR SERVER. While CloudFlare is protecting your server from receiving Heartbleed attacks, you should still upgrade to the latest version of OpenSSL as soon as possible. Get version 1.0.1g here:

If you can’t upgrade immediately, you can recompile OpenSSL with -DOPENSSL_NO_HEARTBEATS.

3. CHANGE PASSWORDS. Even with these fixes, we recommend that you change your password for CloudFlare and any other online services you may use. You should also consider enabling 2-factor authentication, which will help protect your account even if your password is compromised.

MORE INFORMATION AND FAQs. We have documented Frequently Asked Questions (FAQs) here:

CloudFlare was one of the companies originally contacted by the researchers who discovered the bug and has been working closely to ensure that sites are protected. This is a serious issue for the Internet as a whole and, as we learn more details, we’ll continue to update you on actions you can take to protect your online presence.

Thanks again for your support!

The CloudFlare Team

P.S. CloudFlare cryptography expert, Nick Sullivan, will be on a webcast this Wednesday, April 16th, at 10:00AM Pacific, to answer any questions you may have about Heartbleed. Register here:

For the basis of this analysis, we looked for a common Heartbleed pattern: significantly less information is sent to a server than is received. Most of the attacks reference a small heartbeat packet producing a response up to 64k. That’s a huge discrepancy that we can take advantage of.

We grabbed PCAP data for all the suspect IPs in the list and ran those PCAPs through Snort with the signatures found at Emerging Threats and FOX IT. Out of our sample PCAPs, 263 hit on the Snort signatures we used. The full list of IPs we found exhibiting the suspicious behavior is available here: http://pastebin.com/KEX85PYk.

With mass internet surveillance and the indefinite storage of personal data, the risk that a secret key will be collected in the future is a serious concern. Encryption may be giving many users a false sense of security.

The Canada Revenue Agency (CRA) blocked public access to its online services last Tuesday in reaction to the announcement, but that wasn’t fast enough to stop attackers from stealing information, it said on its website.

“Regrettably, the CRA has been notified by the Government of Canada’s lead security agencies of a malicious breach of taxpayer data that occurred over a six-hour period. Based on our analysis to date, Social Insurance Numbers of approximately 900 taxpayers were removed from CRA systems by someone exploiting the Heartbleed vulnerability.”

The CRA said its analysis of the attack is not yet complete and it is continuing to analyze “other fragments of data, some that may relate to businesses” that was also apparently removed.

CSEC asserts that it was not aware of the Heartbleed bug until April 7th, the day the public learned of the bug. However, as CBC points out, that was still one day before the Canada Revenue Agency shut down its website in a belated effort to prevent unauthorized data leakage (Valerie Boyer, “CSEC aware of Heartbleed bug day before CRA website shutdown,” CBC News, 16 April 2014).

A Bloomberg news report claimed that the NSA (and thus, almost certainly, CSEC) has been exploiting the bug for at least two years. The U.S. government has denied that report, claiming that it would have reported the bug if it had been aware of it. It does acknowledge, however, that not all of the cyber security flaws that NSA knows of are disclosed.

“In a case of ‘live by the sword, die by the sword,’ researchers have used the now-infamous Heartlbeed bug in OpenSSL to gain access to black-hat forums. A French researcher named Steven K. is quoted as saying, ‘The potential of this vulnerability affecting black-hat services is just enormous.’ Reportedly, the criminal-minded sites Darkode and Damagelab have already been compromised.”

What went wrong, how the Internet reacted, what we can learn for the future

The Heartbleed vulnerability took the Internet by surprise in April of this year. The vulnerability was one of the most consequential in the history of the Internet, since it allowed attackers to potentially steal login credentials, cryptographic keys, and other private data from up to half of all popular HTTPS sites. In this talk, we take a detailed look at Heartbleed and its aftermath, based on comprehensive measurements and analysis that our research team performed over the past six months. We began tracking Heartbleed’s impact within hours of its disclosure using massive ZMap scans and large network telescopes. This allowed us to track which sites remained vulnerable, observe certificate revocations, and monitor for large scale attacks in close to real time. Based on this data, we also conducted one of the largest ever mass vulnerability notifications, informing the network administrators for all devices still susceptible to Heartbleed across the entire IPv4 address space. Finally, we investigated the question of whether attackers knew about and exploited Heartbleed prior to its public disclosure—and we will present new details about this question in the talk. We hope that by learning from the Heartbleed security disaster, our community can prepare to respond more effectively to such events in the future.

In March 2014, researchers found a catastrophic vulnerability in OpenSSL, the cryptographic library used to secure connections in popular servers including Apache and Nginx. The bug allowed attackers to extract cryptographic keys, login credentials, and other private data from an estimated 22-55% of HTTPS sites. Worsening its severity, the bug was both simple to understand and exploit.

We used ZMap to perform comprehensive scans of the IPv4 address space and popular web servers in the days and months following disclosure. We provide more extensive estimates on who was originally vulnerable, track who patched their sites, and replaced certificates. We will present exactly which server products and devices were vulnerable. We will further discuss how Heartbleed affected the HTTPS CA ecosystem. Worryingly, we find that only 10% of the known vulnerable sites replaced their certificates within the next month, and of those that did, 14% neglected to change the private key, gaining no protection from certificate replacement! We’ll also present the shortcomings in the public key infrastructure that Heartbleed unearthed and problems our community needs to focus on moving forward.

We investigated widespread attempts to exploit Heartbleed post disclosure at four network sites. We will discuss the subsequent exploit attempts we observed from almost 700 sources and the Internet-wide scans that started post disclosure. We also investigated whether exploit attempts took place prior to Heartbleed’s public disclosure, including examining suspicious network traces recorded months earlier. We will disclose new details of these traces and their implications in the talk.

Even with global publicity, Heartbleed patching plateaued after two weeks. To try to help, we notified network administrators responsible for more than 500,000 unpatched systems. While much of the security community (including us!) assumed that mass vulnerability notifications would be too difficult or ineffective, we found that it increased the Heartbleed patching rate by nearly 50%. We will discuss how we performed these notifications, the reactions of network operators, and prospects for performing automatic mass notifications based on Internet-wide scanning in future vulnerability events.

Throughout the talk, we will use real world data to frame what went well and what went poorly in the Internet’s response to Heartbleed. The vulnerability’s severe risks, widespread impact, and costly global cleanup qualify it as a security disaster. However, by understanding what went wrong and learning from it, the Internet security community can be better prepared to address major security failures in the future.
──────────
➤Speaker: Zakir Durumeric
➤EventID: 6321
➤Event: 31th Chaos Communication Congress [31c3] of the Chaos Computer Club [CCC]
➤Location: Congress Centrum Hamburg (CCH); Am Dammtor; Marseiller Straße; 20355 Hamburg; Germany
➤Language: english
➤Begin: Sun, 12/28/2014 23:00:00 +01:00
➤License: CC-by