Linux kernel archives host compromised by attacker

The Linux kernel archive website, which is located at kernel.org, was compromised by attackers last month. According to a statement posted yesterday on the website, unauthorized parties successfully seized root access to several kernel.org servers and planted a trojan. The site hosts the source code of the Linux kernel, and a number of other projects.

The intrusion was reported to kernel.org users earlier this week by site administrator John Hawley. The attack is believed to have occurred on August 12 but wasn't detected until August 28. The attack vector isn't known for certain, but it is thought that the attacker somehow obtained a legitimate user's login credentials and then exploited an unknown privilege escalation vulnerability. The attack was discovered when an Xnest error message was found in the system logs on a server that did not have Xnest installed.

This irregularity prompted further investigation, leading to the discovery of a trojan. The SSH server software on the system was modified and a script to initiate the trojan was found among the system startup scripts. The official statement on kernel.org says that it's still not clear whether the Xnest error message is actually a symptom of the attack or an anomaly.

The kernel.org administrators have responded to the security failure by by taking the affected systems offline and contacting law enforcement authorities. All of the kernel.org servers will be wiped and fully reinstalled. An audit is underway to determine if any of the source or release packages were modified by the attacker. The login credentials and SSH keys of all 448 kernel.org users will also be changed.

The code repositories of the Android Open Source Project (AOSP) are also hosted on kernel.org. Hawley took down the Android code at Google's request after the attack was detected. The AOSP git page currently shows a message explaining the situation and indicating that service could possibly be restored as early as September 1.

The extent of the damage is still not clear, but it's considered highly unlikely that the attacker injected code into the active Linux kernel tree. In a blog post on the Linux.com website, kernel developer and Linux Weekly News writer Jon Corbet published a detailed explanation of how the Linux kernel development workflow, which has multiple layers of code review and relies on distributed version control, poses barriers to such tampering. As Corbet points out, kernel.org is more like a distribution channel for the Linux kernel rather than a hub of development activity.

Although the damage is probably not significant, the incident is still an embarrassment for the Linux kernel development community. This attack occurred one week before the Linux Foundation's annual LinuxCon event, at which the Linux development community celebrated the kernel's 20th anniversary.

Although successful attacks of this nature against Linux development infrastructure are not common, they do occasionally happen. Red Hat servers were compromised in 2008 and a Debian server was compromised in 2006. It serves as a chilling reminder of the breadth of the threat landscape and the challenges of keeping important systems secured against attacks.

People have been screaming it for the past 10 years, but it's becoming increasingly apparent that electronic security is one of the most important fields going forward. The stakes keep getting higher and higher. Our username/password authentications schemes and haphazard approach to finding and closing software exploits are showing time and time again to be inferior security in the face of determined cybercriminals.

The question is why, not whether the servers need to be more secure or not. No matter what security you have there are brains out there that will get around it if they have sufficient motivation.

Are you seriously saying that one don't need to patch security flaws?!?It's like saying "I lost my keys to the house and some thief used them to gain access and steal my things. But Oh well... I don't have to change the lock as the thiefs will gain access no mather what.."

The question is why, not whether the servers need to be more secure or not. No matter what security you have there are brains out there that will get around it if they have sufficient motivation.

Are you seriously saying that one don't need to patch security flaws?!?It's like saying "I lost my keys to the house and some thief used them to gain access and steal my things. But Oh well... I don't have to change the lock as the thiefs will gain access no mather what.."

I don't think Viktor's saying that.

All he's saying is we need to ascertain the motivation behind the attack. Was it just for fun, or something else? These days, we're seeing sophisticated attacks (think RSA/Lockheed Martin/Stuxnet) where its not just defacement.

Understand the motivation and then drive the overall response (e.g. better separation between internal and external-facing servers, etc.). In the meantime, continue to patch, patch, patch!

I wonder if SELinux was enabled on the server. Not saying it is a silver bullet, but it sure makes it much harder when configured properly. Strict policy for the kernel.org distribution channel should be rather elementary in fact.

First off, I have to say, "KUDOS SECURITY TEAM!" Wow, what an awesome disclosure of everything that happened, is happening right now and will be happening in the future! Now that's how you disclose a security breech. Bravo!

Secondly, to those who want to talk about "electronic security" as the importance of the future, no, no it is not.

What is important is EDUCATION OF END USERS. From the sounds of it, this was a social engineering feat of hacking. If you are forced to use a 1024-bit password with key dongle, and someone posing as a cute blonde who's "always been fascinated with computers" wants a gander at the files, you know some punter will be mailing her the dongle and the password.

This is why I use Windows. When's the last time the Windows kernel servers were compromised? Never. 'Nough said.

LOL

I'll bet MS' most important internal source code servers (such as for the kernel) are ridicolously physically secured, NEVER online, have all kinds of logging on with a bunch of people payed to watch it all the time, etc. Or they're online but using VPNs only and ban everything that's unauthorized (not VPN = not allowed, not properly authenticated = banned), still with logging.

The Linux project is a much more open project where most of the code is uploaded over the internet and a lot of everything is done remotely. And of course they do also check the system security periodically, but it appears they should do it more often...

TLDNR: Microsoft's kernel devs are all usually inside MS buildings when coding, this makes it easier to protect yourself from external access compared to the more open Linux kernel development project.

This is why I use Windows. When's the last time the Windows kernel servers were compromised? Never. 'Nough said.

Windows source code's been stolen at least twice (2000 and 2004). Where did the attackers get access to the code from (a quick google turned up empty)?

From Microsoft partners who had copies of the code.Try searching Mainsoft Windows source code leak.

That would have been easier to do if I had remembered mainsoft being the leaker... Searching on variants with MS in the title turned up news articles reporting the initial breach and the code showing up on p2p sites but not indicating how it had happened.

From the sounds of it, this was a social engineering feat of hacking. If you are forced to use a 1024-bit password with key dongle, and someone posing as a cute blonde who's "always been fascinated with computers" wants a gander at the files, you know some punter will be mailing her the dongle and the password.

There was also, at a minimum, a privelege escalation. Having a user account shouldn't make you able to r00t the machine.

SELinux will not neccessarily help if there's a kernel bug that they are able to exploit to raise their privs. Once you are in the kernel, all bets are off for any local security scheme.

I doubt SELinux was on. If you're outside your context, even if you're root, you can't override the access controls. Being "in the kernel" still doesn't help you get past SELinux, unless the bug was in SELinux itself. Even if SELinux didn't stop the attacker, it likely would have provided more and better warning that something bad was going on.

The announcement at kernel.org is worth reading, as it explains the impact (or lack thereof) that the security breach has on the integrity of the kernel source. I wish the article had mentioned that.

Huh? There is a whole paragraph about that...

From the article:

Quote:

The extent of the damage is still not clear, but it's considered highly unlikely that the attacker injected code into the active Linux kernel tree. In a blog post on the Linux.com website, kernel developer and Linux Weekly News writer Jon Corbet published a detailed explanation of how the Linux kernel development workflow, which has multiple layers of code review and relies on distributed version control, poses barriers to such tampering. As Corbet points out, kernel.org is more like a distribution channel for the Linux kernel rather than a hub of development activity.

This is why I use Windows. When's the last time the Windows kernel servers were compromised? Never. 'Nough said.

What do you mean by 'kernel servers'? Do you think they are able to change the Linux source code at all just by having root on these servers? If you don't think that, then why would I be glad I run windows just because of a hiccup in distribution?