Slashdot videos: Now with more Slashdot!

View

Discuss

Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

Hugh Pickens DOT Com writes "Ed Felton writes about an incident, in 2003, in which someone tried to backdoor the Linux kernel. Back in 2003 Linux used BitKeeper to store the master copy of the Linux source code. If a developer wanted to propose a modification to the Linux code, they would submit their proposed change, and it would go through an organized approval process to decide whether the change would be accepted into the master code. But some people didn't like BitKeeper, so a second copy of the source code was kept in CVS. On November 5, 2003, Larry McAvoy noticed that there was a code change in the CVS copy that did not have a pointer to a record of approval. Investigation showed that the change had never been approved and, stranger yet, that this change did not appear in the primary BitKeeper repository at all. Further investigation determined that someone had apparently broken in electronically to the CVS server and inserted a small change to wait4: 'if ((options == (__WCLONE|__WALL)) && (current->uid = 0)) ...' A casual reading makes it look like innocuous error-checking code, but a careful reader would notice that, near the end of the first line, it said '= 0' rather than '== 0' so the effect of this code is to give root privileges to any piece of software that called wait4 in a particular way that is supposed to be invalid. In other words it's a classic backdoor. We don't know who it was that made the attempt—and we probably never will. But the attempt didn't work, because the Linux team was careful enough to notice that that this code was in the CVS repository without having gone through the normal approval process. 'Could this have been an NSA attack? Maybe. But there were many others who had the skill and motivation to carry out this attack,' writes Felton. 'Unless somebody confesses, or a smoking-gun document turns up, we'll never know.'"

Unless somebody has proof that somebody was trying to create a back door then stop with all of the "X-Files" shit. It could have been a hacker trying to put that code in. How was the system that hosted the CVS repository managed? Was it hacked? Was there any investigation or was it possibly somebody that did something stupid and now everybody thinks it's somehow tied to the NSA?!?!?

Let's just go forward with what we know and stop the speculation, that is unless somebody has some hard facts like an IP address that belongs to the government or a chain of evidence.

Considering how b0rked this attempt was, Microsoft actually would be plausible:-). I mean is there any lint or static code analyser who wouldn't have yelled it's ass of on that line? Even GCC does it (not so sure that the version of GCC used in 2003 did this though to be honest).

Obviously you're unaware of how GCC works. Yes, it would have issued a warning message if the line was:wait4: if ((options == (__WCLONE|__WALL)) && current->uid = 0)...

But the author of that little backdoor attempt added the extra 'superfluous' parenthesis around the assignment. Like this:wait4: if ((options == (__WCLONE|__WALL)) && (current->uid = 0))...

And GCC uses those extra parenthesis as an indication that 'The assignment is deliberate and desired. Don't warn me about it. I know what I'm doing'In fact, GCC still uses those extra parenthesis as an indication that the warning message should be suppressed.

That is actually misguided reasoning. If you remove this parenthesis and write it like that
if ((options == (__WCLONE|__WALL)) && current->uid = 0)
the code will fail to compile for a completely different reason. In C (as well as in C++) the assignment operator has very low priority, lower than `&&` operator. That means that the above code would be interpreted as
if (((options == (__WCLONE|__WALL)) && current->uid) = 0)
A code like that would not compile at all, since it attempts to assign 0 to something that is not lvalue. For this reason (and not some "warning"), you actually absolutely need that extra parenthesis.
Also, note that the first condition is also wrapped in parenthesis, which is formally excessive ('==' has higher priority than `&&`, but some users prefer to add that extra parenthesis since they believe it improves readability). For this reason, it is fairly safe to conclude that both parentheses where there from the very beginning. They were not added by that malicious coder.

In 2003, there wasn't even near the IDS/IPS technology of today. Firewalls were in place, but some places still have not moved to internal segments with firewalls on the internal networks.

This could have been anyone. Yes, it -could- have been the Greys, the NSA, the CIA, MI5, Elbonia's secret police, or the Illuminati, but right now, it was a detected hack attempt, and anything more is pure rumor unless there is something definitely specified in the papers Snowden sold to the Guardian.

Thousands of contract employees in the past few years have had access to what Snowden revealed. It is likely that all those "secrets" were old news to every major foreign intelligence service. The NSA is like a vacuum cleaner with a punctured bag: they sweep up info and then leak it like a sieve.

And if Snowden were interested in selling info, he would never have gone public.

Unless somebody has proof that somebody was trying to create a back door then stop with all of the "X-Files" shit. It could have been a hacker trying to put that code in. How was the system that hosted the CVS repository managed? Was it hacked? Was there any investigation or was it possibly somebody that did something stupid and now everybody thinks it's somehow tied to the NSA?!?!?

Yes, there was an investigation [indiana.edu]. The name attached to the log entries belonged to someone who said he didn't make the changes.

It's not a big deal, we catch stuff like this, but it's annoying to theCVS users.

My Favorite from A. Walrond..

Somebody getting access to and inserting exploits directly into the linuxsource is not something we should take lightly. Whilst we understand thelimits of the problem, the fact that it happened at all could get/.'d out ofall proportion and be used to seriously undermine linux's reputation

Note, back in 2003, "/.'d out of all proportion..." which is exactly what this article is all about.

The name attached to the log entries belonged to someone who said he didn't make the changes.

Well, it also had the fact that CVS was just a read-only clone. If you wanted to make a change, you can't submit it to CVS - you had to submit it up the change and eventually it would hit the BitKeeper repo first, then propagate to CVS.

So oddball entries like that mean it not only is a change that doesn't end up back in the BK tree (because there's no pointer back to the BK changeset), so something strange is going on.

Of course, this only affected CVS users - it would not affect BK users (as the CVS was a one-way clone of the BK tree that was autogenerated), so the change not only was only in CVS, but there is no corresponding change in the BK tree to be linked with anyone.

I do listen to the news, so prove to me that it was the NSA rather than some bored college student looking to inject some mayhem? During my first two years of college, I worked as a Systems Operator. We had students who pushed the envelope on APIs, system calls and other things. They'd come up with strange things and bugs and report them to us. "I created this 128 character string and it had somebody's name in it when I printed it out." Well that's what it was like in the late 70s, when memory wasn't

I do listen to the news, so prove to me that it was the NSA rather than some bored college student looking to inject some mayhem?

And why would I seek to prove something to you that it says right in the damned article that nobody knows who did it, or why, or how? I certainly never claimed it was the NSA, and even TFS suggest that, while it could have been the NSA, they don't know.

There is direct proof someone tried to insert a back door, but as far as who did it, nobody fucking knows, and TFS even says that.

Given what the NSA is doing lately, they're a plausible guess, but, there is no proof to suggest what entity did that... NSA, bored college student, the Chinese, aliens, your mom. It says right in the summary they don't know, and unless someone admits to it, they never will -- but nonetheless, code did magically end up in the code repository they couldn't account for and which they caught. So someone did attempt to insert a back door, that much is fact -- the rest of it is speculation, and that's pretty much evident that it is speculation.

You're asking for proof of something that people are only suggesting as a possibility, not claiming as fact. Which means you're not even debating the article, you're debating something the article didn't say but you're acting as if it did.

I would probably be more apt to name the Chinese or the Russians before the NSA, after all, the Chinese have a decent financial stake in linux development, and have had for some time. Red Flag Linux (est. Jan 2000) has it's roots in Chinese state-funded computer science.

Or there was a coding error. After all, one single missing "=" makes its effect that of a back door, even if that had not been the intent.One wonders just how many of this type of "back door" actually exists in the mountain of source code.

The unique thing about this incident is that it was only found by some random diff, not someone reading validly submitted code and saying "oh wait this is a back door".

Given all the massive patches and totally new sections of code that get submitted each year, its apparen

A coding error, which got into CVS and bypassed BitKeeper, for which there's no commit logs to account for it? And which by sheer fluke would also have been an exploit?

Sure, it could have been a coding error, but the description of how they found it and what they couldn't subsequently find would strongly suggest that this got in there by some really, er, 'unusual' mechanism.

It sounds like they did the forensics at the time, and that it didn't come in through any mechanism any of them could account for.

If I understood correctly, this didn't get into CVS from a commit, it just ended up in there. And in many years of working with CVS, I'm pretty sure I've never seen that happen.

The UID assignment was surrounded in parentheses to prevent a compiler warning

strongly suggest that this was a deliberate feature, and not a casual error. I've done a lot of security audits of mission critical systems in my time and I've seen a lot of potentially catastrophic casual errors. You can always see what (innocent) thing the programmer intended to do. There's no 'innocent' thing this change could do. This is intended.

All the criminal activity the NSA has done and continues to do has done nothing but made the entire hardware and software structure of the Internet vulnerable, paving a smooth, superhighway to everyone else in the world that wishes to either destroy modern society or simply steal money from the 99%.

The petard the NSA and Western World will be hoisted upon is one of their own making. (Cylons 1:15)

Actually in this very specific case the Occam razor's version of event was a malicious attempt.

Just the other day I arrived at work to find traces of my externally accessible office door being jimmied. Now what is more likely (1) that the janitors lost the master key and forced their way in to vacuum clean or (2) that someone (likely a petty burglar) opened it and took my missing radio?

There is a suid exploit inserted in an unauthorized way. The simplest explanation is a backdoor attack. The subsequent investigation seems to support this further, even though we still do not know with certainty since the guilty person was never caught. However we can operate on the assumption that this was a malicious attack and discuss it as such.

I think what Virtucon was trying to complain about, but failed miserably in conveying, was not the claim that someone was trying to insert a backdoor into the Linux kernel, but that that someone was the NSA, as there is zero evidence to point to any suspect.

was not the claim that someone was trying to insert a backdoor into the Linux kernel, but that that someone was the NSA, as there is zero evidence to point to any suspect

Except nobody is saying "it was the NSA".

In fact, the last sentence of TFS says

Bould this have been an NSA attack? Maybe. But there were many others who had the skill and motivation to carry out this attack,' writes Felton. 'Unless somebody confesses, or a smoking-gun document turns up, we'll never know.'

> The code snippet is an obvious attempt at a backdoor. What more "proof" do you need?

Actually, the code snippet, without context is not an obvious attempt. It is a cleverly hidden attempt that COULD be a genuine error.

However, having no approved submission to track it back to, that makes it an obvious attempt, even more so with evidence that the hosting system was compromised.

I would even call it obvious if they had an approved submission if the submitter didn't have some good justification for mucking with an error handler for a case that shouldn't happen, those tend to not need much maintenance.

Skip that. How about we go with "Never attribute to malice that which can be explained by stupidity." We're talking about a single character that turns what everyone seems to agree would be an error-check, into an exploit. This sort of thing happens all the time, and nobody says the butler did it, in the observatory, with the lead pipe, when it does. They say "Hey, man, your last diff patch might have a bug." (explains bug)... a few hours to days later, they reply back with "Oh hey thanks for catching that

Except that clearly SOMEONE was trying to create a backdoor. That someone may not have been a government agency, in fact the summary seems pretty clear to me that they are not saying that it WAS a government agency. The summary clearly states that there is currently no way to know who put the backdoor code into the kernel repository.

A backdoor is not necessarily something put in to allow a government agency to access a system. The term "backdoor" refers to any code which intentionally allows an unauthor

Unless somebody has proof that somebody was trying to create a back door then stop with all of the "X-Files" shit.

Yes. Never try to think that anyone would be interested in compromising a very popular OS. Always assume that it was just a simple mistake, and the fact that safety precautions to avoid this kind of mistake were also bypassed: it's always co-incidence. Never suspect anyone, ever. Security by hiding your head in the sand is the best security ever. By the time you realize they chopped your head off, you're already dead, so you have never have anything to worry about.

That's hardly an investigation and even if you read what people were writing they all were pretty much "meh." and "That's why we do what we do." Was the FBI brought in? No. Any charges filed? No. Was it distributed in the wild? No. The system they put in place caught the problem before it made it out there. I'm much less worried about this than the other vulnerabilities and possible exposures that are already there and are sold by those nice assholes in France who make it their job to sell this stuff

You're forgetting that the NSA is in the news right now, which creates an entirely new angle on it.

I was able to get a copy of the original submission:

Ed Felton writes about an incident, in 2003, in which someone tried to backdoor the Linux kernel, a key component of the GNU/Linux operating system. As you know, Apple just released a new operating system called iOS 6. Is it possible that an NSA contractor, paid in Bitcoins raised through an anonymous Kickstarter project to avoid detection, placed an exploit in the new iPhone 5S? And if so, should the government immediately investigate Google who might have used the feature to implement some sort of tracking bug for people using their iPhones in their Teslas?

It's why placing constants on the left of the equality operator is a good idea in C/C++. The whole line then looks suspicious because its constants are on the right, and the first thing you'll think about is bugs involving operator = instead of operator ==. Unfortunately there's a lot of old code that doesn't do this, but it's easy enough for a compiler to issue warnings about operator = in if-statements.

I have often seen this form, not just in C but elsewhere. I always thought it was more a formatting preference for readability (often the variables being tested part of the equation doesn't change, so move the part that does change closer towards the start of the line)

I never considered that it might help catch or mitigate this sort of error. Truely 0 = uid, while also quite bad yet legal C*, doesn't reassign the UID.

* I always thought this was legal, but now that I try it, I find this gives a compile time error - "error: lvalue required as left operand of assignment". Currently gcc is the only compiler I have on hand, so i can't check some of the much older ones.

I think I read the same thing, or something about the same issue. I always remember seeing some examples of this and the odd and what it would lead to. For some reason, I remember C being the language in question, but maybe not.

* I always thought this was legal, but now that I try it, I find this gives a compile time error - "error: lvalue required as left operand of assignment". Currently gcc is the only compiler I have on hand, so i can't check some of the much older ones.

No, assigning to a non-lvalue (where lvalue is defined, essentially, as "something that can be assigned to") has never been legal C or C++.

Nice implication that this is a standard rather than a preference. A lot of new code "doesn't do this" either because a lot of people don't feel like you do, myself included.

Writing expressions backwards hampers the readability of code and only theoretically catches problems. I can count on one hand the times in my career I've suffered this particular fate and I'm not going to pay a price every day for a structural "solution" to a problem I don't have. I've been working with programmers that do this for 20 years and have never found it to be anything but a nuisance. If it works for them, fine, but it doesn't go into my work. I'm not weak but if you are then by all means, use your crutch.

Sometimes the answer is be better at what you do. Once upon a time you could declare a local variable in a switch statement without explicitly limiting scope. Now you must incur the wrath of the nanny compiler or pollute your code with compiler-pacifying turds. This could be made to work right but the same people who think we need shit like this can't be bothered doing a proper job of implementing it. We have attitudes such as yours to thank for crap like that.

At my new job there are debates on what compiler warming levels to adopt. The reason is the release process enforces a no-tolerance policy for warnings and there's a large code base that will never be warning-free at higher levels. This is what you get when you create compiler-enforced "rules" rather than focus on skills and better code quality, particularly when you abdicate to tools you don't control or even understand.

Personally if I was designing a language I'd ban the single = operator and use:= for plain assignment and == for comparison. Compound assignment like += could remain the way they are. Is it extremely unnatural and annoying to write stuff like if (3 == foo ), I'd be a lot more obvious and a lot less likely to be a typo if it had to say if( foo:= 3 ) to do something bad. Of course it could still be a brain fart, but a pretty bad one at that.

In hindsight this is one of the things that I wish Kernighan & Ritchie (the original authors of C) should have considered.

Pascal and Ada both use ":=" as an assignment operation and "=" for testing equality, so this type of error is a non-issue in these languages. Furthermore Pascal (1970) actually predates C (1972) by two years, so it bears consideration why K&R overlooked this possibility.

That said, nearly all modern compilers (incl. GCC) do print a warning if you use a "=" operation in a if() or

Yeah, I'd often thought that the:= operator should have been used instead of = (possibly leaving == as the equality operator and having a single = give a compiler error).

The only problem is that:= *might* look quite similar to != (i.e. doesn't equal). If we were starting again from scratch, of course, we could choose BASIC's <> instead to avoid that problem, but whether it would be a good idea now, I don't know.

The difference between linux and closed source OSs is that on linux you may be able to identify malicious code in the kernel and remedy this situation. For closed source solutions you're truly fucked through and through. You seriously think Microsoft and Apple haven't backdoored their OS ? Just one more reason to stop using closed source software if you value your privacy, your secrets etc...

I hope the Linux team, which has the security of billions of people in their hands, uses far better security than Felton's article implies. (And for all I know it is.)

The excerpt above suggests that someone happened to notice a change that wasn't pointing to an approval record. What if nobody happened to notice? What if the attacker also created an approval record? And was there a serious effort to find the exploit used and close it, and find the perpetrator?

It is monitored more carefully. Notice that the backdoor was only introduced into the CVS copy, which wasn't the official copy used to create kernel releases. It never made it into the official copy in BitKeeper, because to get there it would've had to go through the official review and approval process that would've caught and rejected it. And without making it into the BitKeeper repository it never would've been used by any major distribution, only by developers and private distributions that pulled from the CVS copy because of objections to BitKeeper.

And today even that unofficial copy is gone. With the change to git I believe there aren't any secondary copies in other version-control systems except maybe private ones developers keep for whatever reason which wouldn't be able to feed changes back into the main repository without going through the review and approval process every submission has to go through.

Long and short, the Linux kernel repository's no more vulnerable than the internal repositories for Windows or the Oracle database system, and it's probably less vulnerable. Microsoft or Oracle's repositories will take commits from any random contractor that's been hired to work on the code, regardless of their background or history. The Linux repository... it may accept submissions from anyone, but the degree of review before approving the submission depends heavily on how well the project maintainers know the submitter. The first submissions from someone the maintainer doesn't know are going to be reviewed with a fine-toothed comb and a skeptical eye, and very few black-hats are going to be willing to spend years submitting high-quality code to build up enough of a reputation with the maintainer to be able to get code in with only a cursory review. It's the difference between a development team and a developer community.

very few black-hats are going to be willing to spend years submitting high-quality code to build up enough of a reputation with the maintainer to be able to get code in with only a cursory review.

I think you are underestimating the value to attackers of compromising Linux, and therefore everything that runs on Linux. Paying someone over several years to build a reputation in a community is nothing for a state intelligence budget or even unusual activity (based on my very limited knowledge).

Also, in IT security, insiders are often considered the greatest security threat. Established community members can be compromised; maybe they need mone

Even if the code had been accepted and committed, it would have been some time before it would have started rolling out into systems. How many people do you know who consistently run the latest Linux kernel? The most popular distros are generally (at least) a few months behind on adopting the latest kernel, so even if this was committed next week it would have likely not shown up in widespread use until the middle of next year at the earliest.

And beyond that, the users that use Linux are likely far less interesting to the NSA than they like to tell themselves to be. Enemies of the state don't generally have an interest in running anything other than windows (which they often steal, so the cost is irrelevant).

If it was the NSA, it'd be hard to trace now. Especially as this is going back to 2003, prior to all this excitement. It doesn't matter. It didn't work, and the NSA is suspected of perpetrating more recent attacks at a different level in the chain.

But this example brings up a good point, which is how vulnurable C and C++ code is in general to obfuscation. It is a known security risk and attack vector, but programmers tend to gloss over it, mainly when they can't accept that they are just as capable of makin

Well if it happened in 2003, then we know it cannot possibly be the NSA. After all, we have been told repeatedly by the mainstream media and by reputable unbiased sites such as our beloved slashdot that the government was 1000% righteous and benevolent from 2001-2008 and only became evil after we elected a socialist anarchist fascist liberal hippie far left islamist atheist democratic dictator to the white house. So clearly, the NSA in 2003 could not have been behind an attempt to insert malicious code into the Linux kernel; and if they somehow were then real Americans had nothing to fear about it anyways!

Quoting their web site:"The goal of the contest is to write code that is as readable, clear, innocent and straightforward as possible, and yet it must fail to perform at its apparent function. To be more specific, it should do something subtly evil. "

But even in C, it should (and does using any reasonable compiler, I think) generate a warning.

Personally, I try to code any comparisons involving literals with the literal on the left (e.g. if(0 == foo)... instead of if(foo == 0)...) so that I get a "lvalue required as left operand of assignment" error if I leave one of the equals signs out. And when I intend an assignment, I do it in a separate statement (e.g. foo = 0; if(0 == foo)...).

On a side note, the compiler folks probably ought to change that error

When I first started using Linux I tried to run a process under a service account by name 'dvrservice'. What actually happened was the process launcher parsed the name 'dvrservice' into an integer, value 0, thereby running the public facing network service as root.

fortunently I detected that before anything bad happend.

It would be more secure, I think, if each install generated a random uid for root, so that instead of (uid = 0) or obfuscated (uid = ser

I don't know how many bits a Linux UID is, I think Windows uses GUIDs (128 bits; but not all unique iirc) so I don't know the odds of a collision, but yeah, it would be a problem if you org assigned standard numbers to service accounts or something...

But if you never referred to accounts by number, only name, and left the numbers to the internal systems...

oh, maybe drive the fundies crazy, UID 666.

I read a booklet at my religious aunts house once that claimed the existance of chmod 666 as part of th

666 is a pretty evil permission actually. Making a file world writable is just asking for trouble. I'm also shocked at the prospect that a Jack Chick type pamphleteer would understand Unix permissions. Those people are usually quite technologically challenged.

In most assembly languages, when you compare against a value, you have use a "compare" instruction that effectively does a subtraction, but throws away the result.In most CPUs, there is a flags register with a zero (Z) bit, which is flipped whenever a value is loaded that is zero.When you want to see if something like your accumulator or another register is an arbitrary value like 100, you need to do a "compare 100" and then "branch if equal to whatever..."If it's zero, you can just load the value and skip the compare step. You get a "free compare" when the value you want to compare against is zero.So if the superuser is not zero, there will be a performance penalty.

Besides, this dumb shit is C's fault for using = and == as operators. Pascal had it right with:= being the assignment operator.

If your language returns a boolean from assignment, then it sucks and invites this sort of thing.

if (a = b)... should always be an error.

Good thing C doesn't have Booleans, then.

It helps to remember that C was intended as a fairly thin layer over assembly. In assembly you can set one register to the value of another, then branch-if-not 0, Two instructions on most platforms. Why should that be an error? Of course, one hopes than on most modern compilers it's a warning.

Most compilers do make this at least a warning. It isn't an error because it's a moderately common C-ism to do this in order to assign and check the return value or a function in 1 statement. Particularly if the value is a standard 0 on error or NULL on error return.

I always use '-Wall -pedantic' for gcc, and if my code is producing any warnings, I always fix them all.

If the kernel developers had been doing this, they would have seen a big fat warning. For those who still like to use this dubious idiom, putting double parentheses around the assignment make the side effects more explicit to the reader and disables the warning.

It likely should be an error if you're setting it to a constant. That's equivalent to forcing true or false in the if, with a side-effect of changing the value of a variable. I can only see a few very convoluted cases where that would make any sense, and there are far more readable ways to write them.

The old 'if (fp = fopen("foo", "r") )' makes sense, but there's no real reason to write it that way any more. It should at least be a warning.

It's an ok to deal with the fact that the BSD socket API is overly verbose and a pain to deal with. The last time I made a mistake accidentally doing = instead of == was a long, long time ago, though. So I'm not sure it's a real problem.

Or did they bypass the normal submission means and somehow just sneak it into an about to be built code block?

Well, did you even read the entire summary? Where it says that pretty much everything was bypassed?

But some people didn't like BitKeeper, so a second copy of the source code was kept in CVS. On November 5, 2003, Larry McAvoy noticed that there was a code change in the CVS copy that did not have a pointer to a record of approval. Investigation showed that the change had never been approved and, stra