Who rooted kernel.org servers two years ago, how did it happen, and why?

More than two years after unknown hackers gained unfettered access over multiple computers used to maintain and distribute the Linux operating system kernel, officials still haven't released a promised autopsy about what happened.

The compromise, which began no later than August 12, 2011, wasn't detected for at least 16 days, a public e-mail and interviews immediately following the intrusion revealed. During that time, attackers were able to monitor the activities of anyone using the kernel.org servers known as Hera and Odin1, as well as personal computers belonging to senior Linux developer H. Peter Anvin. The self-injecting rootkit known as Phalanx had access to a wealth of sensitive data, possibly including private keys used to sign and decrypt e-mails and remotely log in to servers. A follow-up advisory a few weeks later opened the possibility that still other developers may have fallen prey to the attackers.

For three weeks in September and early October, officials kept kernel.org closed so the servers that run it could be rebuilt. When the site reopened on October 4, a message on the front page prominently warned of the breach and noted the steps taken to rebuild the site. "Thanks to all for your patience and understanding during our outage and please bear with us as we bring up the different kernel.org systems over the next few weeks," the message concluded. "We will be writing up a report on the incident in the future."

Almost two years later, the report has yet to be delivered. The promise to deliver an incident report remained on kernel.org as recently as March 1 of this year, before being quietly pulled the following day. To this day, officials have yet to provide key details, including exactly how many machines were compromised, how the attackers were able to gain root access to them, and what they did once they seized control. The delay contrasts sharply with autopsies that were delivered promptly following twosimilar compromises of Apache.org, the official distributor of the open-source Apache Web server.

"As a user, I think everyone should be a little bit disappointed they didn't execute that transparency in a follow-up," Dan Rosenberg, a senior security researcher at Azimuth Security, told Ars. Without a thorough autopsy, "it's hard to really know what level of negligence was involved in the compromise."

Linux developer and maintainer Greg Kroah-Hartman told Ars that the investigation has yet to be completed and gave no timetable for when a report might be released. He said officials remain confident of preliminary findings that the attackers were not able to tamper with the source code that millions of organizations use to compile their Linux systems.

"We went through many rounds of validation of the kernel releases on the site, regenerating them from the Git tree and old backups," he wrote in an e-mail. "All of them were fine, nothing was found to be tampered with or touched at all."

Git is the name of the system that tracks changes made to the source code for the Linux kernel. It uses a series of 160-bit cryptographic hashes to account for the revisions. Copies of the repository and all changes are then cached in thousands of locations around the world. A mismatched hash in one or more location would quickly indicate unofficial changes. Kroah-Hartman also told Ars kernel.org systems were rebuilt from scratch following the attack. Officials have developed new tools and procedures since then, but he declined to say what they are.

"There will be a report later this year about site [sic] has been engineered, but don't quote me on when it will be released as I am not responsible for it," he wrote.

While there's no evidence that backdoors or other malicious code were surreptitiously inserted, the 2011 breach of one of the world's most important software development organizations should nonetheless remain a concern. That's especially true now that we're living in an era where actors of powerful nation-states have been known to hijack Microsoft's official Windows update mechanism and deliberately weaken cryptographic coding standards. Transparency is more important than ever. If Linux maintainers expect trust, they should make good on their promise to tell users precisely what happened and how.

Promoted Comments

Yep. It's also possible that an underfunded, understaffed org has to prioritize resources and has decided that it is more important to move forward than to figure out exactly what happened. In the ideal world they'd do both, but if you think they're obligated to be idealists... please send kernel.org your resume and number of hours you're prepared to invest figuring out a two-year-old breach that they believe has been totally fixed.

Not everything is a conspiracy, even these days.

Making security a low priority is reason enough to be disappointed and concerned. It doesn't have to be a conspiracy.

The professional response after a long wait would be to say that they still don't have enough data or are unsure, not to just remain silent.

Kind of like not calling your bank to let them know you're going to be 2 weeks late on a payment; It's not respectful.

I can get behind that sentiment.

All we can do is intuit a rationale from kernel.org's silence, which is likely to be inaccurate. I doubt they're hiding anything horrendous, so my intuition leads me to believe that they don't have anything like a solid answer, and nobody wants to put their name on a response that is half guessing and half reading tea leaves.

I agree that something is better than nothing here, and maybe Ars's coverage will shake something loose.

You are utterly missing the point. NOBODY owes you a bleeding thing, nothing. You haven't paid for it, you (probably) haven't contributed time or anything else. How is it you are owed jack all? Linux is just people that write some software.

There are plenty of people who do contribute code to the kernel and who trusted those behind kernel.org to provide a proper, secure service in return. They have been failed and they do deserve an explanation of exactly what went wrong and what steps have been taken to prevent it happening again.

Personally I suspect at this point the root cause is known and it's down to poor security practices by one or more of the "big names" in kernel development. It's the only explanation, short of "they have no idea", that explains the reluctance to reveal their findings. And either of those frankly highlights some very concerning issues with how the kernel team treat security.

63 Reader Comments

Disclosure: This is not a Windows vs Linux comment. I have more Linux machines than Windows and I love Linux.

But this is exactly what im more afraid about Linux. Being that its so dispersed, there are to many ways to infect and compromise servers. This just talks about kernel.org but how about mirror servers and those that never went public? There are thousands if not mirror servers and Linux servers to download updates from. How do you even know if one of this servers is secure? Its not compromised? Imagine how many Linux machines would get infected !

This happen for example to ProFTD and it was a disaster because the infection was downloaded to so many servers we cannot even count.

This is my biggest fear about Linux. Updates, patches, etc, are downloaded from anywhere and you can´t possible know if these are safe or not. Even if you by maintain your own update servers, I would say that most don´t.

So far, Microsoft update servers where never compromised, in decades Microsoft never send a bogus update to Windows users, which of course would also be catastrophic if their update servers are compromised downloading bogus data to users worldwide.

In this regard open source fails miserably, since there so many entry points you cannot control that are multiple servers worldwide your Linux machines could be downloading a Trojan horse tomorrow being hidden as an update.

Disclosure: This is not a Windows vs Linux comment. I have more Linux machines than Windows and I love Linux.

But this is exactly what im more afraid about Linux. Being that its so dispersed, there are to many ways to infect and compromise servers. This just talks about kernel.org but how about mirror servers and those that never went public? There are thousands if not mirror servers and Linux servers to download updates from. How do you even know if one of this servers is secure? Its not compromised? Imagine how many Linux machines would get infected !

This happen for example to ProFTD and it was a disaster because the infection was downloaded to so many servers we cannot even count.

This is my biggest fear about Linux. Updates, patches, etc, are downloaded from anywhere and you can´t possible know if these are safe or not. Even if you by maintain your own update servers, I would say that most don´t.

So far, Microsoft update servers where never compromised, in decades Microsoft never send a bogus update to Windows users, which of course would also be catastrophic if their update servers are compromised downloading bogus data to users worldwide.

In this regard open source fails miserably, since there so many entry points you cannot control that are multiple servers worldwide your Linux machines could be downloading a Trojan horse tomorrow being hidden as an update.

Because the code and repositories are signed and there is a solid web-of-trust among the main developer's GPG keys.

@nibb are you talking about the base system or in general about all packages? The base system is compiled by the distro managers. For example, in the case of Ubuntu, Canonical is responsible for that, and it's up to them to make sure that their code is not compromised. if you're talking about third party software then you are right. But, that's because how package managers work. You need admin rights for every package you install.

I believe that Canonical is trying to avoid that with their click packages for the Ubuntu phone. It's still questionable what they're going to do on the desktop.

Disclosure: This is not a Windows vs Linux comment. I have more Linux machines than Windows and I love Linux.

But this is exactly what im more afraid about Linux. Being that its so dispersed, there are to many ways to infect and compromise servers. This just talks about kernel.org but how about mirror servers and those that never went public? There are thousands if not mirror servers and Linux servers to download updates from. How do you even know if one of this servers is secure? Its not compromised? Imagine how many Linux machines would get infected !

This happen for example to ProFTD and it was a disaster because the infection was downloaded to so many servers we cannot even count.

This is my biggest fear about Linux. Updates, patches, etc, are downloaded from anywhere and you can´t possible know if these are safe or not. Even if you by maintain your own update servers, I would say that most don´t.

So far, Microsoft update servers where never compromised, in decades Microsoft never send a bogus update to Windows users, which of course would also be catastrophic if their update servers are compromised downloading bogus data to users worldwide.

In this regard open source fails miserably, since there so many entry points you cannot control that are multiple servers worldwide your Linux machines could be downloading a Trojan horse tomorrow being hidden as an update.

Because the code and repositories are signed and there is a solid web-of-trust among the main developer's GPG keys.

Followup: If your distribution isn't signing its packages, you should change distributions. If it is, you know who to blame.

QFT. I only get software from and install updates from signed RPMs who's signature I can validate to a trusted source, like Red Hat. Note how all the various package managers automate this stuff. If you use apt-get, yum, urpmi, or something else, you can't install packages or updates that don't have good signatures without at least responding to a file requester or setting a command line flag. Your package manager will also only work with a set of repositories you have configured.

Of course the question then is how does Red Hat know that the packages they include in RHEL/FC are good? Clearly one of the things they will do is work with kernel.org to maintain the source for the kernel. They could of course security audit their own separate copy of the source, but it would be pointless. This is why each of the vendors has people they pay to work on the kernel and other vital projects.

The professional response after a long wait would be to say that they still don't have enough data or are unsure, not to just remain silent.

Kind of like not calling your bank to let them know you're going to be 2 weeks late on a payment; It's not respectful.

I can get behind that sentiment.

All we can do is intuit a rationale from kernel.org's silence, which is likely to be inaccurate. I doubt they're hiding anything horrendous, so my intuition leads me to believe that they don't have anything like a solid answer, and nobody wants to put their name on a response that is half guessing and half reading tea leaves.

I agree that something is better than nothing here, and maybe Ars's coverage will shake something loose.

Disclosure: This is not a Windows vs Linux comment. I have more Linux machines than Windows and I love Linux.

But this is exactly what im more afraid about Linux. Being that its so dispersed, there are to many ways to infect and compromise servers. This just talks about kernel.org but how about mirror servers and those that never went public? There are thousands if not mirror servers and Linux servers to download updates from. How do you even know if one of this servers is secure? Its not compromised? Imagine how many Linux machines would get infected !

This happen for example to ProFTD and it was a disaster because the infection was downloaded to so many servers we cannot even count.

This is my biggest fear about Linux. Updates, patches, etc, are downloaded from anywhere and you can´t possible know if these are safe or not. Even if you by maintain your own update servers, I would say that most don´t.

So far, Microsoft update servers where never compromised, in decades Microsoft never send a bogus update to Windows users, which of course would also be catastrophic if their update servers are compromised downloading bogus data to users worldwide.

In this regard open source fails miserably, since there so many entry points you cannot control that are multiple servers worldwide your Linux machines could be downloading a Trojan horse tomorrow being hidden as an update.

I do not think there is anything to fear about how linux updates generally arrive at client machines. Anyone can correct me if I am out to lunch. Please note that I am writing the following for those who may not be aware, and I certainly not saying you are not aware how general updating of LInux machines works. Most distros use Repositories or Repos which are generally mirrors of packages and sources which are generally signed/verified by keys created by the individual package providers. These signed packages are then uploaded to mirrors, to help distribute the load of supplying these packages across the world. Thus through webs of trusts, and as long as the signing and verification has not been broken, regardless of what mirror a client pulls from, these packages should be whole and untouched if they pass the verification. Mirrors could be compromised, but the package verification process on the client side should detect any altered packages under normal circumstances. Most distros will push the required keys for distro recognized repos as part of its install. That being said, the trust falls on the multitude of developers who provide these updates. Whether you are using Microsoft products or Linux products, the risks are generally the same. As for Microsoft Updates server not being compromised, I wish that were the case, but as been noted in several recent high profile problem in Europe and Middle East, that may not been the case. I am not even going into the whole NSA problem.

A delay this long either means incompetence in the investigation or the whole story is being slowrolled because its worse than we were told.

I'm leaning towards the latter.

It's also possible that they don't have enough information to say with any certainty. If it wasn't detected for 16 days, and they only keep logs for two weeks, there won't be much data to determine exactly what happened.

I don't know that's the case here, but it wouldn't be unheard of.

Yep. It's also possible that an underfunded, understaffed org has to prioritize resources and has decided that it is more important to move forward than to figure out exactly what happened. In the ideal world they'd do both, but if you think they're obligated to be idealists... please send kernel.org your resume and number of hours you're prepared to invest figuring out a two-year-old breach that they believe has been totally fixed.

Not everything is a conspiracy, even these days.

I completely agree that not everything is a conspiracy. That doesn't mean software developers who expect our trust should be excused from providing an autopsy about a breach that was so potentially serious. How expensive or time-consuming can it be to issue a report like this?

Again, no one is saying they were hacked by a nation state or a criminal gang. Knowing what we know now, though, we certainly can't rule it out, at least until the kernel.org people come clean.

Responsibility? Its a HOBBY, this is a VOLUNTEER ORGANIZATION, not some sort of business. If you want to go in and volunteer to complete this report we'll all appreciate it, but the fact is that any project undertaken by kernel.org or any other such organization only happens because it is interesting enough for someone to take on and do. Alternately some business could pay for it or put some people on it. Looks to me like nobody is volunteering to do it, and nobody has decided to pay for it to be done. Nobody is obligated to do anything, nor has some responsibility to do anything. You can always not use Linux if you don't like the way it is developed and supported. Its as simple as that. Anyone here who's using Linux and isn't willing to contribute to necessary or useful kernel.org projects can basically STFU, eh? Honestly, I'm not trying to start a flame war with anyone, but it bears pointing out now and then that Linux is Linus Torvald's hobby project, not some product you pay for...

Seriously? It's 2013 and there are still people thinking that Linux development relies solely on the spare time of hobbysts from their basements... Wake up! Linux is big business and gets maintained by increasingly more bigtime corporate developers, who are being paid to do so.

The Linux Foundation took upon itself the responsability to be among the ones promoting and protecting the operating system, and get a lot of money from donations of big IT companies. Check the link below for more info on this: http://en.wikipedia.org/wiki/Linux_Foundation#Members

Seriously? It's 2013 and there are still people thinking that Linux development relies solely on the spare time of hobbysts from their basements... Wake up! Linux is big business and gets maintained by increasingly more bigtime corporate developers, who are being paid to do so.

The Linux Foundation took upon itself the responsability to be among the ones promoting and protecting the operating system, and get a lot of money from donations of big IT companies. Check the link below for more info on this: http://en.wikipedia.org/wiki/Linux_Foundation#Members

You are utterly missing the point. NOBODY owes you a bleeding thing, nothing. You haven't paid for it, you (probably) haven't contributed time or anything else. How is it you are owed jack all? Linux is just people that write some software. It is NOT A BUSINESS. There are people who are paying because they DO get benefits, and those people can legitimately decide they want this or that done. The rest of us? We can comment, we can donate our time to get things done we want to see done, or we can just use the software that others have generously licensed for our use and stop deluding ourselves that we're owed anything at all. You aren't Red Hat, SUSE, etc that has paid are you? If you are paying LF something then by all means complain to them that you want X, Y, or Z attended to. Otherwise? GET REAL, you aren't owed anything at all, PERIOD, END OF REPORT. Any thought you have to the contrary is pure delusion.

Seriously? It's 2013 and there are still people thinking that Linux development relies solely on the spare time of hobbysts from their basements... Wake up! Linux is big business and gets maintained by increasingly more bigtime corporate developers, who are being paid to do so.

The Linux Foundation took upon itself the responsability to be among the ones promoting and protecting the operating system, and get a lot of money from donations of big IT companies. Check the link below for more info on this: http://en.wikipedia.org/wiki/Linux_Foundation#Members

You are utterly missing the point. NOBODY owes you a bleeding thing, nothing. You haven't paid for it, you (probably) haven't contributed time or anything else. How is it you are owed jack all? Linux is just people that write some software. It is NOT A BUSINESS. There are people who are paying because they DO get benefits, and those people can legitimately decide they want this or that done. The rest of us? We can comment, we can donate our time to get things done we want to see done, or we can just use the software that others have generously licensed for our use and stop deluding ourselves that we're owed anything at all. You aren't Red Hat, SUSE, etc that has paid are you? If you are paying LF something then by all means complain to them that you want X, Y, or Z attended to. Otherwise? GET REAL, you aren't owed anything at all, PERIOD, END OF REPORT. Any thought you have to the contrary is pure delusion.

Except it IS a business or rather a non-profit consortium and the people who run it and work for the Linux Foundation are employees that get paid. I think it is you that is delusional on this topic.

Also, the more capitals you use the less serious people will take you.

Seriously? It's 2013 and there are still people thinking that Linux development relies solely on the spare time of hobbysts from their basements... Wake up! Linux is big business and gets maintained by increasingly more bigtime corporate developers, who are being paid to do so.

The Linux Foundation took upon itself the responsability to be among the ones promoting and protecting the operating system, and get a lot of money from donations of big IT companies. Check the link below for more info on this: http://en.wikipedia.org/wiki/Linux_Foundation#Members

You are utterly missing the point. NOBODY owes you a bleeding thing, nothing. You haven't paid for it, you (probably) haven't contributed time or anything else. How is it you are owed jack all? Linux is just people that write some software. It is NOT A BUSINESS. There are people who are paying because they DO get benefits, and those people can legitimately decide they want this or that done. The rest of us? We can comment, we can donate our time to get things done we want to see done, or we can just use the software that others have generously licensed for our use and stop deluding ourselves that we're owed anything at all. You aren't Red Hat, SUSE, etc that has paid are you? If you are paying LF something then by all means complain to them that you want X, Y, or Z attended to. Otherwise? GET REAL, you aren't owed anything at all, PERIOD, END OF REPORT. Any thought you have to the contrary is pure delusion.

Except it IS a business or rather a non-profit consortium and the people who run it and work for the Linux Foundation are employees that get paid. I think it is you that is delusional on this topic.

Also, the more capitals you use the less serious people will take you.

Call it whatever you like, you're not paying for it, so it doesn't owe you squat. Pretty simple, got it?

You are utterly missing the point. NOBODY owes you a bleeding thing, nothing. You haven't paid for it, you (probably) haven't contributed time or anything else. How is it you are owed jack all? Linux is just people that write some software.

There are plenty of people who do contribute code to the kernel and who trusted those behind kernel.org to provide a proper, secure service in return. They have been failed and they do deserve an explanation of exactly what went wrong and what steps have been taken to prevent it happening again.

Personally I suspect at this point the root cause is known and it's down to poor security practices by one or more of the "big names" in kernel development. It's the only explanation, short of "they have no idea", that explains the reluctance to reveal their findings. And either of those frankly highlights some very concerning issues with how the kernel team treat security.

Personally I suspect at this point the root cause is known and it's down to poor security practices by one or more of the "big names" in kernel development. It's the only explanation, short of "they have no idea", that explains the reluctance to reveal their findings. And either of those frankly highlights some very concerning issues with how the kernel team treat security.

I agree. If not an error by a big name, then the NSA.

And if it was the NSA few insiders would know, just the person national security letter was delivered. a lawyer and any technical person(s) essential to carrying out the act. Remember you cannot even tell the CEO or other managers if you get a national security letter, just the company lawyer. Same in the UK.

Call it whatever you like, you're not paying for it, so it doesn't owe you squat. Pretty simple, got it?

Who named you spokesperson? Your views certainly don't represent all kernel developers.

Wait you think all the people spending lots of their time and energy on the project, wouldn't appreciate his "It's only a hobbyist sideproject, if you expect professionalism please use another OS, we don't owe you anything" attitude?

Call it whatever you like, you're not paying for it, so it doesn't owe you squat. Pretty simple, got it?

Who named you spokesperson? Your views certainly don't represent all kernel developers.

Wait you think all the people spending lots of their time and energy on the project, wouldn't appreciate his "It's only a hobbyist sideproject, if you expect professionalism please use another OS, we don't owe you anything" attitude?

No idea how you came up with that idea!

I think you guys have taken this far beyond what I've said. Nobody OWES you anything. That doesn't mean that you can't hope for something, and that you can't say "well, Linux is better if we get X, Y, and Z." I mean that's what the community is built on is meeting each other's and our own needs and desires. It is just the way that people state it as "We are owed this, that and the other by Linux Foundation!" that I object to. If you put it as you would like to see something and you're willing to facilitate that thing happening (and in all fairness some people have stated they are), then great! If at that point you have issues with the support you get from LF members, or other members if you happen to be one yourself, of course that's another thing (again some people have expressed that they have such issues, that's fine).

Honestly, if the governance of LF seems bad and insufficient my advice is to look at who's running it and how they got there and see about changing things. AFAIK anyone can join LF here https://www.linuxfoundation.org/about/j ... idual/join and from there you can run for a seat on the board, etc. Certainly members have a far better claim to being owed some sort of level of response from LF and are in a MUCH better position to have their concerns addressed than some random guy bitching on Ars about things. Remember, ultimately Linux success rests with the community. I just think that claiming to be 'owed' something doesn't play well with people who have actually spent good chunks of their lives giving you a world-class OS.

Remember, ultimately Linux success rests with the community. I just think that claiming to be 'owed' something doesn't play well with people who have actually spent good chunks of their lives giving you a world-class OS.

But you're still ignoring the fact that there are lots of people who have contributed to the community in all manner of ways and they absolutely deserve to be told what happened. They are "owed" and explanation and they should get it.

You can't say "Linux success rests with the community" and simultaneously defend those who screwed up from having to explain what they did wrong and how they're guaranteeing it doesn't happen again. Those involved in kernel.org are only a small part of the community and they owe the rest an explanation. It's that simple.

I think you guys have taken this far beyond what I've said. Nobody OWES you anything.

What you don't seem to be understanding is that the goal of the Linux Foundation is to get more people to use Linux. And the only way to achieve this is to NEVER use the "it's for free so what do you expect?" argument. People are trusting their personal data to their OS, businesses trust their whole business to their IT and expect it to work.The only reason people use Linux is because they have become to expect good engineering and reliability from it. This trust is what makes Linux successful, hiding behind "It's for free" only harms this and doesn't help anybody and if people working on the Kernel had this attitude Linux would never gotten anywhere.

The Linux Foundation certainly doesn't wants to create an image of "Yeah we may not care about your data and security, but hey it's for free, so still a good deal, right?" and that means handling serious security breaches with the professionalism and urgency they deserve.

You are aware that the whole paragraph before this sentence was about how they wouldn't care about some "random guy bitching on Ars" and how you should get a seat on the board to get your criticism taken seriously? I really hope you see the irony there.

I think you guys have taken this far beyond what I've said. Nobody OWES you anything.

What you don't seem to be understanding is that the goal of the Linux Foundation is to get more people to use Linux. And the only way to achieve this is to NEVER use the "it's for free so what do you expect?" argument. People are trusting their personal data to their OS, businesses trust their whole business to their IT and expect it to work.The only reason people use Linux is because they have become to expect good engineering and reliability from it. This trust is what makes Linux successful, hiding behind "It's for free" only harms this and doesn't help anybody and if people working on the Kernel had this attitude Linux would never gotten anywhere.

The Linux Foundation certainly doesn't wants to create an image of "Yeah we may not care about your data and security, but hey it's for free, so still a good deal, right?" and that means handling serious security breaches with the professionalism and urgency they deserve.

You are aware that the whole paragraph before this sentence was about how they wouldn't care about some "random guy bitching on Ars" and how you should get a seat on the board to get your criticism taken seriously? I really hope you see the irony there.

No, I don't see any irony actually. I certainly don't claim to be some big wheel in the community, but I've been here since day one and I HAVE contributed in small ways. I don't think kernel.org or the LF in general is the be-all and end-all of Linux frankly. I was here long before it existed, and Linux was a perfectly fine OS which we've been using for commercial purposes since 1994, even a bit earlier. So honestly, I don't see any one incident, person, or even organization, as possessing such a vital role in the whole thing that it is somehow vital that it do this or that. Linux is a volunteer tinkerer/hacker community, and we're perfectly fine here thanks.

If you feel that people who happen to supply you with free software as a result of what they do OWE you something, you're deluded, that's all there is to it. Would it be better if kernel.org released a report on the hack? Yeah, in an ideal world. OTOH if its a choice between that and kernel work that makes the OS more useful to me? I am far from sure I care that much about a report on a 2 year old hack. Nobody owes me such a report, I haven't offered to help produce it, and I don't happen to have an active LF membership, so LF actually owes me nothing, has other shit on its plate no doubt, and really Linux will survive and thrive without everything in the world being perfect and ideal.

Honestly, MOSTLY, Linux got where it is because it was fun to work on. I sure as hell don't blame people for not wanting to spend their time doing some boring hack report because you feel entitled to it. If some business that feels it needs more reassurance about Linux wants to pay a guy to go over to kernel.org and make a report, GREAT! Otherwise people should go on doing their fun stuff and not worry about it. If I want to buy RHEL from Red Hat and pay for the support and expect them to have engineered a high confidence in the integrity of that product into it, then I will. Sometimes I do. Interestingly Red Hat doesn't seem to feel it is necessary to report on the hack or their guys over at LF would be doing it. We can of course disagree on how that makes us feel.

If they don't know wtf happened by now they will never know, and in any case it is WAY too late to have prevented untold manner of other related security issues.

When the FreeBSD build cluster was compromised, it was shut down and kept out of action until the cause of the compromise was determined (compromise of private key), impact was determined and the opportunity was taken to retire support for some legacy components (cvsup) while they were at it.

You really don't see the irony in first claiming that one's opinion only matters if they're on the board of directors and in the next sentence try to tell anybody how important the community is? Really?

And you still don't get it - one of the main points of the Linux Foundation is to increase Linux's outreach, a goal it shares with lots of commenters on Ars. Making security a "further runs" bullet point is incredibly damaging to this overall goal which in turn damages the work all those volunteers do. Luckily the largest part of the developer base doesn't share this opinion and considers security flaws in the kernel itself highest priority - now this same mentality should also be applied to other parts of the project. And yes if they find security problems in the kernel they actually look into how it ended up there and how to avoid this problem in the future - for very good reasons.

You seem to be fine with considering security not really important ("Ah I think I prefer new features to understanding the security flaws in the current system thank you"), which just makes me really, really hope I'll never have to use any software you've worked on for important stuff.

Yeah I'm sure all those servers and users running Linux are doing that just because it's so much fun to hack on it. 99.9% of all people using Linux will never contribute a single line of code to the kernel or even look at the source code, they use Linux for some much more pragmatic reasons than that. Good security being one of them.

Call it whatever you like, you're not paying for it, so it doesn't owe you squat. Pretty simple, got it?

Who named you spokesperson? Your views certainly don't represent all kernel developers.

Wait you think all the people spending lots of their time and energy on the project, wouldn't appreciate his "It's only a hobbyist sideproject, if you expect professionalism please use another OS, we don't owe you anything" attitude?

No idea how you came up with that idea!

I think you guys have taken this far beyond what I've said. Nobody OWES you anything. That doesn't mean that you can't hope for something, and that you can't say "well, Linux is better if we get X, Y, and Z." I mean that's what the community is built on is meeting each other's and our own needs and desires. It is just the way that people state it as "We are owed this, that and the other by Linux Foundation!" that I object to. If you put it as you would like to see something and you're willing to facilitate that thing happening (and in all fairness some people have stated they are), then great! If at that point you have issues with the support you get from LF members, or other members if you happen to be one yourself, of course that's another thing (again some people have expressed that they have such issues, that's fine).

Honestly, if the governance of LF seems bad and insufficient my advice is to look at who's running it and how they got there and see about changing things. AFAIK anyone can join LF here https://www.linuxfoundation.org/about/j ... idual/join and from there you can run for a seat on the board, etc. Certainly members have a far better claim to being owed some sort of level of response from LF and are in a MUCH better position to have their concerns addressed than some random guy bitching on Ars about things. Remember, ultimately Linux success rests with the community. I just think that claiming to be 'owed' something doesn't play well with people who have actually spent good chunks of their lives giving you a world-class OS.

I think you are over-analyzing this situation. When you join a group or a forum or anything on the net, there are certain rules or TOS guidelines that you have to follow, or else your access may get disabled or even terminated. Ars has a TOS. On the flip side, there is a reasonable expectation that when you give a service personally identifying information they assume responsibility for that information. You can say "It doesn't owe you anything" until you are blue in the face and that doesn't change the fact that they do have a responsibility to safeguard information that they request or accept. With that said, if this was Sony there would be lawsuits, not just griping. The LF is losing credibility over this.