Critical Linux vulnerability imperils users, even after “silent” fix

For more than two years, the Linux operating system has contained a high-severity vulnerability that gives untrusted users with restricted accounts nearly unfettered "root" access over machines, including servers running in shared Web hosting facilities and other sensitive environments. Surprisingly, most users remain wide open even now, more than a month after maintainers of the open-source OS quietly released an update that patched the gaping hole.

The severity of the bug, which resides in the Linux kernel's "perf," or performance counters subsystem, didn't become clear until Tuesday, when attack code exploiting the vulnerability became publicly available (note: some content on this site is not considered appropriate in many work environments). The new script can be used to take control of servers operated by many shared Web hosting providers, where dozens or hundreds of people have unprivileged accounts on the same machine. Hackers who already have limited control over a Linux machine—for instance, by exploiting a vulnerability in a desktop browser or a Web application—can also use the bug to escalate their privileges to root. The flaw affects versions of the Linux kernel from 2.6.37 to 3.8.8 that have been compiled with the CONFIG_PERF_EVENTS kernel configuration option.

"Because there's a public exploit already available, an attacker would simply need to download and run this exploit on a target machine," Dan Rosenberg, a senior security researcher at Azimuth Security, told Ars in an e-mail. "The exploit may not work out-of-the-box on every affected machine, in which case it would require some fairly straightforward tweaks (for someone with exploit development experience) to work properly."

The fix to the Linux kernel was published last month. Its documentation did not mention that the code patched a critical vulnerability that could jeopardize the security of organizations running Linux in highly sensitive environments. This lack of security advisories has been standard practice for years among Linus Torvalds and other developers of the Linux kernel—and has occasionally been the subject of intense criticism from some in security circles.

Now that a fix is available in the kernel, it will be folded into all of the affected stable kernel releases offered by kernel.org, which maintains the Linux core code. Individual distributions are expected to apply the fix to their kernels and publish security updates in the coming days.

Additional details of the bug are available here, here, here, and here. People running vulnerable machines with untrusted user accounts should check with their distributors to find out when a patch will be available and what steps can be taken in the meantime. One user of a Red Hat Linux distribution posted temporary mitigation steps here, although at time of writing, Ars was unable to confirm that they worked. Readers are encouraged to post other mitigation advice in comments.

Promoted Comments

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

Unfortunately this is quite true. The Linux devs have a policy of saying anything could be a security bug, so they actively downplay known security bugs, not marking them as such in most cases. The user namespace vulnerability from a couple weeks ago (which could likewise get you root) also wasn't reported as a security issue, just passed off as a generic bug fix.

There are many, many examples (just see what the PaX Team or Brad Spengler write about to see many other instances), but for whatever reason the Linux devs don't take security seriously.

The fix to the Linux kernel was published last month. Its documentation did not mention that the code patched a critical vulnerability that could jeopardize the security of organizations running Linux in highly sensitive environments. This lack of security advisories has been standard practice for years among Linus Torvalds and other developers of the Linux kernel—and has occasionally been the subject of intense criticism from some in security circles.

And how many people are going to end up in jail, tortured, or dead, because they relied on the security of Linux to protect them from bad guys, including bad governments?

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

People can die because of this stuff, now. It's not 1993 anymore.

You're sensationalizing your argument. Yes, cyber security is important, but Linux is still one of the most secure systems around. There are flaws, much like anything man-made, but certainly less frequent than other systems (aside from FOSS BSD derivatives). You're lying to yourself if you think Windows/OSX don't have gaping security flaws. Plus, data encryption is still a thing.

That being said, you honestly think Linux devs are actively trying to cover up this stuff? The code is freely available. Anyone can see it. Anyone can get on a mailing list and see the code for newly patched vulnerabilities. If you weren't too busy pointing fingers and blaming people, and knew anything about analyzing code, you could have spotted this flaw and patched it two years ago. There are 10 million lines of code in the Linux kernel. Join the community and help out!

The Linux kernel developers are notorious for not documenting security fixes. Here's an instance from a couple weeks ago. A security issue was fixed, but it wasn't documented as such, which simply leaves people guessing. Brad Spengler has been very vocal about this issue, and has found many, many patches that were pushed to the mainline that were to fix security vulnerabilities, yet weren't documented as security fixes. He's not the only one, but he has a fairly long track record of actually discovering vulnerabilities in the Linux kernel (as well as the creator of the grsecurity patchset).

The Linux kernel developer's rationale is that any bug could potentially be security issue, so singling out some as 'security issues' will cause distros/users to be selective with their fixes... unfortunately that rationale doesn't match reality, where the distros are selective with what they backport regardless... now they just backport even less, letting known-vulnerabilities remain because the kernel devs didn't label the patch as a fix, nor attach the CVE if it had one.

I think it's pretty safe to assume that there are many local root exploits available. I wouldn't ever run untrusted code outside of a sandbox as it's pretty easy to set up seccomp (whitelist of system calls) + a container (separate process/network namespace) for something like a programming competition or an evalbot. You can easily restrict a program to only having read/write/exit and restrict the resources it can use with the container's cgroup.

Of course, there are many people using Linux, FreeBSD and other operating systems as multi-user machines, but I really wouldn't give someone unsandboxed access to a machine without trusting them.... the kernel just has too much code, especially all of the drivers.

Might be worth directly referencing the the actual vulnerability ID, it's CVE-2013-2094. According to the RedHat discussion, SELinux does not provide protection against this, as the exploit can disable it with ring0 shellcode. I see a lot of people saying though that it won't work in at least some virtualized environments as well. Hopefully patches get pushed out quickly, it's definitely unfortunate that the Linux devs don't at least ref CVEs themselves.

One thing to note is that while Red Hat Enterprise Linux 6 does not have a kernel version that is in the "range", the code that creates the vulnerability was backported into the 2.6.32 kernel that is used. So RHEL6 and all derivatives/rebuilds (CentOS, Scientific Linux, PUIAS, Oracle Enterprise Linux, CloudLinux, various others) are vulnerable as well.

I suspect that if the fix had been marked as being a security fix then the work to backport it would be further along. The flip side is that a widely published exploit is also more likely to have appeared earlier. The latter is really not as bad as it seems -- it's likely that this vulnerability was already in use by private blackhat exploits, although that's not provable by their very nature.

The fix to the Linux kernel was published last month. Its documentation did not mention that the code patched a critical vulnerability that could jeopardize the security of organizations running Linux in highly sensitive environments. This lack of security advisories has been standard practice for years among Linus Torvalds and other developers of the Linux kernel—and has occasionally been the subject of intense criticism from some in security circles.

And how many people are going to end up in jail, tortured, or dead, because they relied on the security of Linux to protect them from bad guys, including bad governments?

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

The fix to the Linux kernel was published last month. Its documentation did not mention that the code patched a critical vulnerability that could jeopardize the security of organizations running Linux in highly sensitive environments. This lack of security advisories has been standard practice for years among Linus Torvalds and other developers of the Linux kernel—and has occasionally been the subject of intense criticism from some in security circles.

And how many people are going to end up in jail, tortured, or dead, because they relied on the security of Linux to protect them from bad guys, including bad governments?

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

Unfortunately this is quite true. The Linux devs have a policy of saying anything could be a security bug, so they actively downplay known security bugs, not marking them as such in most cases. The user namespace vulnerability from a couple weeks ago (which could likewise get you root) also wasn't reported as a security issue, just passed off as a generic bug fix.

There are many, many examples (just see what the PaX Team or Brad Spengler write about to see many other instances), but for whatever reason the Linux devs don't take security seriously.

Might be worth directly referencing the the actual vulnerability ID, it's CVE-2013-2094. According to the RedHat discussion, SELinux does not provide protection against this, as the exploit can disable it with ring0 shellcode. I see a lot of people saying though that it won't work in at least some virtualized environments as well. Hopefully patches get pushed out quickly, it's definitely unfortunate that the Linux devs don't at least ref CVEs themselves.

The fix to the Linux kernel was published last month. Its documentation did not mention that the code patched a critical vulnerability that could jeopardize the security of organizations running Linux in highly sensitive environments. This lack of security advisories has been standard practice for years among Linus Torvalds and other developers of the Linux kernel—and has occasionally been the subject of intense criticism from some in security circles.

And how many people are going to end up in jail, tortured, or dead, because they relied on the security of Linux to protect them from bad guys, including bad governments?

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

People can die because of this stuff, now. It's not 1993 anymore.

You're sensationalizing your argument. Yes, cyber security is important, but Linux is still one of the most secure systems around. There are flaws, much like anything man-made, but certainly less frequent than other systems (aside from FOSS BSD derivatives). You're lying to yourself if you think Windows/OSX don't have gaping security flaws. Plus, data encryption is still a thing.

That being said, you honestly think Linux devs are actively trying to cover up this stuff? The code is freely available. Anyone can see it. Anyone can get on a mailing list and see the code for newly patched vulnerabilities. If you weren't too busy pointing fingers and blaming people, and knew anything about analyzing code, you could have spotted this flaw and patched it two years ago. There are 10 million lines of code in the Linux kernel. Join the community and help out!

So is the argument that if they report known security bugs as such that only the bugfixes they state are a security problem will get backported to the distro kernels? That would seem almost logical to me, if they'd state that. Unfortunately I think the way they are doing it is only encouraging the distros to not bother porting the mainline kernel out of compatibility fears. For example I can compile a mainline kernel on my Centos 6.3 VM and it boots and runs just fine, but when I go to play around with Samba I find that mount.cifs doesn't like share security in the 3.8.8 kernel I just made. So, since there are both unlabeled security bugs, the occasional regression, and linux distributions that operate mostly as support providers and would like to limit their costs, the kernel languishes an entire major version number back.

The fix to the Linux kernel was published last month. Its documentation did not mention that the code patched a critical vulnerability that could jeopardize the security of organizations running Linux in highly sensitive environments. This lack of security advisories has been standard practice for years among Linus Torvalds and other developers of the Linux kernel—and has occasionally been the subject of intense criticism from some in security circles.

And how many people are going to end up in jail, tortured, or dead, because they relied on the security of Linux to protect them from bad guys, including bad governments?

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

People can die because of this stuff, now. It's not 1993 anymore.

You're sensationalizing your argument. Yes, cyber security is important, but Linux is still one of the most secure systems around. There are flaws, much like anything man-made, but certainly less frequent than other systems (aside from FOSS BSD derivatives). You're lying to yourself if you think Windows/OSX don't have gaping security flaws. Plus, data encryption is still a thing.

That being said, you honestly think Linux devs are actively trying to cover up this stuff? The code is freely available. Anyone can see it. Anyone can get on a mailing list and see the code for newly patched vulnerabilities. If you weren't too busy pointing fingers and blaming people, and knew anything about analyzing code, you could have spotted this flaw and patched it two years ago. There are 10 million lines of code in the Linux kernel. Join the community and help out!

The Linux kernel developers are notorious for not documenting security fixes. Here's an instance from a couple weeks ago. A security issue was fixed, but it wasn't documented as such, which simply leaves people guessing. Brad Spengler has been very vocal about this issue, and has found many, many patches that were pushed to the mainline that were to fix security vulnerabilities, yet weren't documented as security fixes. He's not the only one, but he has a fairly long track record of actually discovering vulnerabilities in the Linux kernel (as well as the creator of the grsecurity patchset).

The Linux kernel developer's rationale is that any bug could potentially be security issue, so singling out some as 'security issues' will cause distros/users to be selective with their fixes... unfortunately that rationale doesn't match reality, where the distros are selective with what they backport regardless... now they just backport even less, letting known-vulnerabilities remain because the kernel devs didn't label the patch as a fix, nor attach the CVE if it had one.

I don't mean this in an insulting way, but as an actual question: were you an adult yet, in 1993? It was a very different time. Few (no?) governments had massive electronic surveillance programs aimed at their own citizens, and they certainly weren't buying exploits on the black market to break into computers of citizens they really wanted to spy on.

2013 is frighteningly different, but the Linux kernel devs don't seem to have internalized just how critical what they're doing is becoming. Twenty years ago, people didn't generally bet their lives on Linux. Today, at least if they're dissidents of any kind, they are probably doing so, even if they don't know they're using it.

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

Unfortunately this is quite true. The Linux devs have a policy of saying anything could be a security bug, so they actively downplay

Perhaps they just have a little historical perspective having used Unix for more than 5 minutes or having used more than one form of Unix in their lives. This kind of bug is not news and not something to get hysterical about or fodder for bashing a particular flavor of Unix.

That being said, you honestly think Linux devs are actively trying to cover up this stuff?

They absolutely are! Deliberately and consciously. It is the slimiest goddamn thing ever.

And they're completely disingenuous about it, too, saying, "well, we can't really tell if there's a security implication blahblahblah", when they knew all along that there was.

What the critics are asking for is dead simple: if a developer knows a patch is to fix a security flaw, say so. That's all. They don't want super hyper mega analysis, they just want to know if a patch is known to cover a known security vulnerability. It's the simplest request in the world... and the kernel devs are absolutely adamant that not only will they not do this, that they will actively hide security vulnerabilities.

It is bullshit, bullshit, bullshit. I would cheerfully throw a drink in the face of every one of them.

I don't mean this in an insulting way, but as an actual question: were you an adult yet, in 1993? It was a very different time. Few (no?) governments had massive electronic surveillance programs aimed at their own citizens, and they certainly weren't buying exploits on the black market to break into computers of citizens they really wanted to spy on.

2013 is frighteningly different, but the Linux kernel devs don't seem to have internalized just how critical what they're doing is becoming. Twenty years ago, people didn't generally bet their lives on Linux. Today, at least if they're dissidents of any kind, they are probably doing so, even if they don't know they're using it.

Speaking as someone who was an adult (at least in age) in 1993, and had already served a hitch in the Navy, I can tell you that 'security breaches' are not limited to "cyber security".

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

Unfortunately this is quite true. The Linux devs have a policy of saying anything could be a security bug, so they actively downplay

Perhaps they just have a little historical perspective having used Unix for more than 5 minutes or having used more than one form of Unix in their lives. This kind of bug is not news and not something to get hysterical about or fodder for bashing a particular flavor of Unix.

This situation isn't really news, but only because silent fixes are still the norm with Linux kernel developers. Not all, but a huge number of them try to hide security issues. It's reckless and irresponsible. Exploit authors spot them, but distro maintainers are much less likely to... which means the fixes aren't getting out to the users, only the blackhats.

Vulnerabilities aren' the problem (regardless of how good you are, some will slip through)... the problem is their response to them.

I mean once an attacker has complete control over any data and other actual work the computer is used for, may all the gods prevent him from changing my precious setup files. Yes, it is vastly harder to detect intrusion once you are rooted, but the idea that a sysadmin's time is allvaluable and a user's time is not, especially on an OS where they are often one and the same, is painfully obsolete.

I mean once an attacker has complete control over any data and other actual work the computer is used for, may all the gods prevent him from changing my precious setup files. Yes, it is vastly harder to detect intrusion once you are rooted, but the idea that a sysadmin's time is allvaluable and a user's time is not, especially on an OS where they are often one and the same, is painfully obsolete.

This is a critical threat for any server where you allow not-fully-trusted users to login (which many web hosts do, or corporate networks).

On the other hand, for a personal desktop/laptop system... you're right, this isn't really a big deal, since Linux's (and Windows/OS X's) security model largely does nothing to help you for the most part.

Obligatory XKCD:

This is a much harder issue that has only really started to be explored in the last few years (mostly thanks to Smart Phones). Mandatory Access Control schemes like SELinux & App Armor help a bit, but only a bit. We've seen better approaches from Android (each app runs with its own UID/GID), as well as iOS/OSX/Windows Store (heavy sandboxing... not sure if they do the same user thing).

Those systems, though, are still very young, and need more more time to mature. Not only that, the systems need to give better controls to the user about what apps can do (Android really should let users disable any privilege an app requests... that would be a huge step forward).

But once those systems mature, then things like this issue would be critical as well, possibly more so because people will be very willing to run arbitrary applications released by third parties.

Security is not a game anymore. It's probably the most important thing in computing. Yet, the Linux devs actively try to hide critical vulnerabilities, because they don't like security researchers.

Unfortunately this is quite true. The Linux devs have a policy of saying anything could be a security bug, so they actively downplay known security bugs, not marking them as such in most cases. The user namespace vulnerability from a couple weeks ago (which could likewise get you root) also wasn't reported as a security issue, just passed off as a generic bug fix.

There are many, many examples (just see what the PaX Team or Brad Spengler write about to see many other instances), but for whatever reason the Linux devs don't take security seriously.

Reducing the visibility of security problems in their product makes it look better when someone who takes them at their (false) word compares it to a competing product that is highly upfront about when it resolves exploits.

That being said, you honestly think Linux devs are actively trying to cover up this stuff?

They absolutely are! Deliberately and consciously. It is the slimiest goddamn thing ever.

And they're completely disingenuous about it, too, saying, "well, we can't really tell if there's a security implication blahblahblah", when they knew all along that there was.

What the critics are asking for is dead simple: if a developer knows a patch is to fix a security flaw, say so. That's all. They don't want super hyper mega analysis, they just want to know if a patch is known to cover a known security vulnerability. It's the simplest request in the world... and the kernel devs are absolutely adamant that not only will they not do this, that they will actively hide security vulnerabilities.

It is bullshit, bullshit, bullshit. I would cheerfully throw a drink in the face of every one of them.

So what do you want? You want the kernel devs to advertise that the bug exists so that people can exploit it before the downstream vendors get it out the door? Every OS maker holds back information on security flaws. And if you read Red Hats bugzilla, there is quite a bit of discussion on the issue and where it is actually a vulnerability. There are tons of bugs that could be used as a security hole IF certain conditions are met. For some, the conditions may never occur. You can't expect the kernel devs to track down all permutations of the code they produce.

That being said, you honestly think Linux devs are actively trying to cover up this stuff?

They absolutely are! Deliberately and consciously. It is the slimiest goddamn thing ever.

And they're completely disingenuous about it, too, saying, "well, we can't really tell if there's a security implication blahblahblah", when they knew all along that there was.

What the critics are asking for is dead simple: if a developer knows a patch is to fix a security flaw, say so. That's all. They don't want super hyper mega analysis, they just want to know if a patch is known to cover a known security vulnerability. It's the simplest request in the world... and the kernel devs are absolutely adamant that not only will they not do this, that they will actively hide security vulnerabilities.

It is bullshit, bullshit, bullshit. I would cheerfully throw a drink in the face of every one of them.

So what do you want? You want the kernel devs to advertise that the bug exists so that people can exploit it before the downstream vendors get it out the door? Every OS maker holds back information on security flaws. And if you read Red Hats bugzilla, there is quite a bit of discussion on the issue and where it is actually a vulnerability. There are tons of bugs that could be used as a security hole IF certain conditions are met. For some, the conditions may never occur. You can't expect the kernel devs to track down all permutations of the code the produce.

There are channels to distribute that information to the appropriate people (such as the security@ mailing list), but that's, quite often, simply not done. Instead, you're leaving a situation where code that is obviously a security vulnerability is just silently added to the kernel. spender has repeatedly said how he can just search through the commit logs for various key words and generally always find many bugs that are trivially exploitable, with just a glance.

Remember, many people critical of this policy are kernel developers as well that actively contribute to the Linux kernel.

Anything could potentially be a security issue... but some things are obviously a security issue.

Not only that, the systems need to give better controls to the user about what apps can do (Android really should let users disable any privilege an app requests... that would be a huge step forward).

I would REALLY like that to be implemented.I'm quite security/privacy conscious, but when I read what permissions almost all apps request ... it's mind boggling. Either you wave away your security/privacy or don't install apps.Implementation of your suggestion would enhance ppl's security/privacy enormously!

...Anything could potentially be a security issue... but some things are obviously a security issue....

I think this is an important point that most people here are forgetting. Lets also be realistic, people keep stating that the "silent report" issue is because the developers stance is that any bug can have unknown security implications, therefore they should technically all be security bugs.

Now, its a rather extreme viewpoint, I also don't quite agree with it myself. However I cant find anything in it I think anyone can say isn't factual? Especially when dealing with a kernel any bug really can be a security bug.

Now, onto the current predicament, reading the detailed explanation of how this bug is exploitable it is entirely within the realms of possibility that particular route hadn't occurred to whoever submitted this and they just thought "huh, that might be bad, I am going to fix that". Seriously, do we know that didn't happen? If that is how this patch came into existence, then its likely no one involved in its creation thought "oh no a massive security problem of earth shattering proportions". If there is one thing that the internet has taught us, it's things that some people look at and think "that is obviously a problem because of X" other people don't even contemplate the existence of X.

I don't like the silent update viewpoint myself, but I'm not going to insult the integrity and ethics of people who have proven themselves knowledgeable and useful to the kernel development community because they have a controversial viewpoint. We wouldn't like people insulting our integrity for saying software patents are bad would we? That is certainly a controversial and not universally held belief.

There are channels to distribute that information to the appropriate people (such as the security@ mailing list), but that's, quite often, simply not done. Instead, you're leaving a situation where code that is obviously a security vulnerability is just silently added to the kernel. spender has repeatedly said how he can just search through the commit logs for various key words and generally always find many bugs that are trivially exploitable, with just a glance.

Remember, many people critical of this policy are kernel developers as well that actively contribute to the Linux kernel.

Anything could potentially be a security issue... but some things are obviously a security issue.

Hmm. Let's see: They fixed it. They didn't tell anyone it was a security fix. This means that if you keep your systems up to date as a matter or course you get this as soon as you can (even without reading about it elsewhere), while those who are looking for exploits aren't notified that there's a glaring hole in all machines that aren't updated. It's a very mild form of security by obscurity if I'm not totally wrong.

You can argue that this is not the best way to handle this (while others may argue it IS the best way to handle this), but it surely isn't the worst. The worst would be announcing the problem without having a fix. Or not announcing it and still having no fix.

How often does this happen (unannounced security fixes)? Because I'm not updating production machines just for the sake of it, so knowing which updates are important security-wise and which are not is useful. But if this happens all the time anyway I can just update all the time and be done with it.

Or this is just another facet of all the chaos and informality that Linux reeks with. Well, it's Linux and a bazaar isn't a cathedral.

But as others already said: If you have users on your machine, you're fucked anyway. Or even more, if you use computers with any large mainstream OS you're doomed. Security will never be a solved problem in anything that has more than a few KB (maybe MB if you're really serious) of code.