Heartbleed Fixes Taking Longer as Websites Plug Gaps

By Jordan Robertson -
Apr 14, 2014

Websites afflicted by the Heartbleed security flaw are finding that it’s taking longer than anticipated to recover from the fallout.

Heartbleed, which can expose people to hacking of their passwords and other sensitive information, sent companies rushing to patch their systems after the security flaw came to light last week. What some didn’t foresee was the time and cost needed to restore user data and fix interruptions caused by suppliers and partners.

Team Snap Inc., like many other Internet companies vulnerable to Heartbleed, sought to plug the vulnerability with a software update and minor technical adjustments, yet soon discovered that wasn’t enough. Team Snap’s hosting company, which provides their Internet infrastructure, caused a breakdown when it applied its own fix and disrupted customer websites.

That scenario illustrates the hidden costs faced by individuals and businesses as they seek to fix one of the biggest security threats in Internet history, said Michael Shaulov, chief executive officer and co-founder of Lacoon Security Ltd., a mobile-security company based in San Francisco.

“Just take the salary of all the people in IT and security and divide it by one week -- that’s probably for everyone, everyone across the board,” Shaulov said in a telephone interview. “There is a ripple effect.”

Heartbleed is one of the biggest security flaws to hit the Internet. The bug, which was discovered by researchers from Google Inc. and a Finnish company called Codenomicon, affects OpenSSL, a type of open-source encryption.

Ripple Effects

Some BlackBerry Ltd. (BBRY) software, including its BBM messaging service for iOS and Android, is affected and the company is working on fixes, it said in an April 10 blog post. BlackBerry smartphones and tablets aren’t compromised, the company said. Calls to BlackBerry’s corporate offices weren’t immediately returned yesterday.

Bloomberg News reported Friday that the National Security Agency has known about the bug for two years and exploited it as a basic part of its spying toolkit. The Office of the Director of National Intelligence denied that the agency was aware of the vulnerability before 2014.

Hacking Attacks

Two days after applying the fix, Boulder, Colorado-based Team Snap, whose sports website has 6 million registered users, encountered disruptions. Photos that people had uploaded of their children’s sports teams suddenly stopped rendering, and they couldn’t upload any more. Leagues and clubs that pay the company to run team Web pages saw their logos and information disappear.

Team Snap’s entire staff of 43 was involved in getting the website to work again, notify customers and change passwords, said Ken McDonald, vice president of customer acquisition.

“It definitely snowballed, and I don’t think any of us when it first happened imagined how many people would be touched in so many ways,” McDonald said. “It’s almost as though you’re in neutral. We have this long list of things that customers want to improve, and instead of doing that you’re just patching and communicating what’s been going on.”

Yahoo Inc. (YHOO) found some of its users’ information spilled onto the Internet after its website was found to vulnerable to the Heartbleed bug a day after its public disclosure.

“As soon as we became aware of the issue, we began working to fix it,” the Sunnyvale, California-based company said in an e-mailed statement April 9.

Tracy Kellmer, a spokeswoman for Bryn Mawr, said most the fixes were applied in minutes and that systems were down only briefly.

While businesses and governments usually rush to apply software patches to defuse security threats, consumers notoriously make the worst choice of all: Doing nothing. Almost six years after the Conficker worm emerged, exploiting a programming flaw in Microsoft Corp. (MSFT)’s Windows operating system, the program is still infecting computers.

A major flaw in the Domain Name System that governs Web addresses uncovered by security researcher Dan Kaminsky in 2008 has been mostly neutralized because the companies patched the flaw quickly.

Rapid Response

Heartbleed takes more steps to fix. The bug concerns a programming error in OpenSSL, which protects information flowing between servers and customers’ computers. Left unaddressed, the flaw allows hackers to spy on private communications and extract the data from computers with compromised connections.

While early estimates placed the bug inside potentially hundreds of millions of websites, subsequent inquiry revealed a far lower figure. Before Heartbleed was disclosed publicly on April 7, just half a million websites had it and were vulnerable to attack, according to Netcraft Ltd., a U.K.-based cyber-security firm.

Large websites such as Google and Facebook Inc. pounced on the issue and plugged any Heartbleed security gaps. Smaller and medium-sized businesses are taking longer, potentially exposing sensitive information.

Industry Response

The security industry’s response to the bug went exactly as anticipated, according Pat Peterson, co-founder and CEO of Agari Data Inc., a San Mateo, California-based e-mail security company.

Fixing vulnerable Android devices will require investments by handset makers and wireless carriers, and companies that haven’t updated will test the patch and ensure it won’t disrupt their systems, Peterson said. He compared it to distributing a new vaccine.

“Certainly it would be easy to get to health-care workers in developed countries,” Peterson said. “But how about packaging it up and getting it to Sub-Saharan Africa or the jungles of Brazil. The supply chains in those countries need to be able to reliably get the vaccine to every nook and cranny.”