Summary
In his 17-minute keynote at O'Reilly OSCON Java 2011, Martin Odersky gives one of the more succinct and clear explanations I've seen on why shared-memory concurrency is virtually impossible to get right.

Years ago at one of the CodeMash conferences, Brian Goetz gave a more in-depth presentation on the subject. This was, I think, a little before he published Java Concurrency in Practice. In his presentation, he made a very convincing case that showed how nearly-impossible it is to write correct code for shared-memory concurrency (Java threads, in his speech). More importantly, even if you do get it right it's incredibly fragile, so that any change in your code can easily introduce new concurrency bugs, and because testing is not possible, you don't know when these bugs appear.

For some reason, Brian didn't want the talk to be filmed or otherwise distributed. This is unfortunate because it was a great explanation of the difficulties; from my own experiences I was already mostly convinced that shared-memory concurrency was the wrong path but Goetz' talk finished the process.

> So when it is impossible to get Java concurrency right,> almost all Java programs are broken, simply by using> threads? This would surprise me.

Well, many Java programs do not use threading, so those are probably OK. And in my experience (>10 yrs doing professional Java programming full time) it really IS true that nearly 100% of Java programs using threading are "broken".

Note that "broken" means it is possible for them to experience unintended behavior when the timing of threading is especially unlucky. Which could, for instance, result in one out of every 10,000 runs doing things wrong because of threading issues. Honestly, most Java programs have enough OTHER bugs in them that the threading bugs get mixed in with the background of other issues. But the threading bugs ARE there, and if you are trying to write bug-free code then using shared-memory threading is almost certainly going to prevent that unless you invest an obscene amount of time and specialized expertise into it.

> Well, many Java programs do not use threading, so those> are probably OK. And in my experience (>10 yrs doing> professional Java programming full time) it really IS true> that nearly 100% of Java programs using threading are> "broken".> > Note that "broken" means it is possible for them to> experience unintended behavior when the timing of> threading is especially unlucky. Which could, for> instance, result in one out of every 10,000 runs doing> things wrong because of threading issues. Honestly, most> Java programs have enough OTHER bugs in them that the> threading bugs get mixed in with the background of other> issues. But the threading bugs ARE there, and if you are> trying to write bug-free code then using shared-memory> threading is almost certainly going to prevent that unless> you invest an obscene amount of time and specialized> expertise into it.

Outstanding explanation. Unfortunately the threading learning curve is steep and long, and to get to the point where you understand the fragility is no small undertaking. So most seem stuck at this level of disbelief.