That Was the Breach That Was

By Jack M. Germain
Sep 27, 2011 5:00 AM PT

A series of intrusions into the heart of the Linux Kernel.org servers in late August that went undetected for some 17 days is still shrouded in mystery. If Linux developers know how the breach occurred, they are not saying. They also are keeping mum on the extent of any damage the break-in caused.

The attackers apparently compromised the servers the Linux Kernel.org uses to maintain and distribute the Linux operating system. A second breach was discovered on Sept. 8 that compromised the servers for Linux Foundation infrastructure including LinuxFoundation.org, Linux.com and their subdomains.

Details about the cause and purpose of the Linux server breaches still are not available. But some security analysts suspect that the attackers grabbed Linux developers' usernames, email addresses, passwords and personal information stored with the accounts.

"This is a big deal. It is not wise to think of it otherwise. Over the last year, the bad guys have been increasingly targeting parts of the information and data foundation that can be penetrated through multiple attacks," Jeff Schmidt, CEO of JAS Global Advisors, told LinuxInsider.

More to It

Linux is a big part of the underpinnings of much of what we do on the Web. So by going after Linux.org, the attack struck at the heart of our computing processes, Schmidt agreed. But he felt the attack goes beyond that scenario.

Both Kernel.org and the related Linux Foundation.org servers were attacked separately, though it's likely the incidents are related. Both sites were still down reportedly for maintenance weeks after the attacks were discovered. The long delay in returning the multiple related Linux Web sites to service suggests that a more serious law enforcement investigation is taking place, he said.

Anybody who went through all that trouble to break into Kernel.org and modify the source code would have to expect they would soon be visited by the police, he explained.

"I do not think that the Linux source code was the intended target. I think the target was the credentials for the key kernel developers.

Reported Damage Sparse

Linux Foundation officials declined to provide direct responses about the break-ins. The official view suggested that the Linux organization was attacked by happenstance and not as an intended target.

"Unfortunately, all our resources are focused on restoring service right now. We'd be happy to answer any remaining questions after services are back up. It is important to note that this wasn't an attack on Linux. Unfortunately, servers are compromised all the time," Jennifer Cloer, director of communications and community for The Linux Foundation, told LinuxInsider.

The Linux Kernel.org staff found a Trojan on two critical servers that resulted from an intrusion no later than Aug. 12, according to an email that John Hawley, chief Kernel.org administrator, sent to users at Kernel.org.

"Files belonging to ssh (openssh, openssh-server and openssh-clients) were modified and running live. These have been uninstalled and removed, all processes were killed and known good copies were reinstalled," he wrote.

It remains unknown what vector was used to launch the attack. But the attackers had gained root access level privileges, he concluded. With root access, the attackers were able to gain entry system-wide.

Lucky Strike

"This is a big wake-up call. This breach could have been astronomically worse. If the attack had been carried out with more sophistication, the attackers could have done a lot worse damage than they did. The gut feeling is that it is more of an accidental intrusion," Joseph Steinberg, cybersecurity expert and CEO of Green Armor Solutions, told LinuxInsider.

Originally, it was a malware type of breach, according to Steinberg. Part of the danger is that now that people know the site can be breached, worse things could happen next time. That point was seconded by Lance James, director of intelligence at Vigilant, a security consultancy that provides strategic guidance across industries.

"I think a lot of these attacks are opportunistic. Once the bad guys find out that Linux.org is vulnerable, they continue to attack. Once you have an opportunity, you take advantage of that. I doubt that this was an intended targeted attack on Linux.org," James told LinuxInsider.

Expect Fallout

Steinberg likened the Kernel.org breach to a hypothetical similar attack on the Microsoft automatic update servers. Such a breach, if it happened with the Windows OS, would infect hundreds of thousands of Windows PCs worldwide.

"What happened with the Linux breach is not quite the same thing because the updates are not automatically being pushed out. But we were very lucky," he said. "But pushing out a phony kernel update could happen. We need to bolster the site's security to prevent that."

Once Pandora's Box is opened, you can never pull that modified kernel back in, warned Steinberg. The fact that the altered code is out there is a problem. The fact that it was out there for so long before being discovered is a problem. Hopefully the Linux community will be quick to fix it.

"The long-term message here is that if we are breached we can quickly put code back. The message has to be that this won't happen again," he said.

Linux Isn't broken

This was not the first and only attack to hit the Linux OS, James said. Over time, Linux servers and other Linux computers get hit.

The breach-- one or more of them -- will not scare enterprise or individual users away, asserted James. We don't see businesses and customers fleeing Microsoft software every time a breach happens to that platform, he said.

"You mitigate the problem, and then you carry on. I don't think it will affect major decisions not to use Linux or not. As an example, often people don't buy insurance until after a rock comes through the window. Typical Mac unsers don't invest in virus protection because there has been little need for it. Macbooks don't come with antivirus. The same holds true for how people view the security of Linux. You should always treat your computing environment with an eye towards better security," said James.

Linux Malware Exists

James has a good repository of Linux malware that he acquired in his role as a security expert. It is much smaller than what he has collected from attacks of Microsoft Windows. But Linux malware does exist, as do rootkits and everything else, he said.

"It is just a different agenda to attack Linux. The point of entry is different. But that doesn't mean that you shouldn't be using antivirus or rootkit checkers or integrity checkers, etc. In some ways you want to do that more because Linux is often being used as a server," explained James.

One theory is that Linux is safer than other OSes because it has so many different distros, so malware writers can't target them all universally. But James rejects that theory of security by obscurity. He does not see that as a fair assessment.

"Linux runs from the same Linux kernel. Distros use different windows managers, that's all. When you look at virus advisories for Linux, you see that they run across multiple Linux distros.

Early Warning

So is the latest round of attacks to the Linux heartland anything more than a bump in the road? That question has two sides to it, James believes.

"We've seen a lot of breaches this year. We are learning from them. On the other side, people become complacent that nothing is secure," he said.

As long as these breaches are manageable incidents, we do not have to view the breach as the worst thing in the world. The bigger concern is not the breach but what comes out of it. For instance, explained James, are passwords exposed?

"I think what scares most people today is the stuff that we don't know about these breeches. What worries me more is what are we not hearing about these breeches. What has been breached that we don't know about yet? There has to be a ton of them, I'm sure," he concluded.

Jack M. Germain has been writing about computer technology since the early days of the Apple II and the PC. He still has his original IBM PC-Jr and a few other legacy DOS and Windows boxes. He left shareware programs behind for the open source world of the Linux desktop. He runs several versions of Windows and Linux OSes and often cannot decide whether to grab his tablet, netbook or Android smartphone instead of using his desktop or laptop gear.