Beautiful concurrency

I am writing a chapter for a book called "Beautiful code", edited by Greg Wilson. The chapter is a tutorial about Software Transactional Memory in Haskell.

After a lot of useful feedback, I have now completed Version 2 of the paper. The Haskell code is also available.

I would welcome any comments or questions you have on the paper, or constructive suggestions for improving it; the more concrete the better. You can see what people said about version 1 on the version 1 talk page: Talk:SantaClausProblem .

The book is aimed at a general audience of programmers, not Haskell geeks, so I have tried to explain everything necessary as I go along. So if you are not a Haskell expert, your input would be particularly valuable to me.

If you give your real name somewhere in your text (or email it to me), I'll add you to the acknowledgements at the end of the chapter. Notably, I'd like to acknowledge Steven807, Tibbe, Fanf, Garious, Rycee, Brecknell, Mvanier, Gaal, Fernando, Gknauth, EngineerScotty, BoAdler.

Simonpj 14:26, 22 December 2006 (UTC) To add a note, begin with four tilde signs ~~~~; the Wiki will fill in your user name and date.

Sylvain 13:45, 11 January 2007 (UTC) small points: since you go through explaining that the do-notation is overloaded and works for STM as well as IO (page 8), you might want to tell the same for return on page 11 (return being given with a type a -> IO a on page 5). There is also a double "Here is the code" at the bottom of page 17.

Brecknell 13:59, 11 January 2007 (UTC) The rewrite of the paragraph at top of page 17 has left behind a fragment that I think just needs to be deleted: "just runs each of the actions in sequence".

Brecknell 14:15, 11 January 2007 (UTC) Last sentence of page 18: "Here is another way approach that..." Delete either "way" or "approach".

page 5, top of page: For example, here are two Haskell functions -> For example, here are the type signatures for two Haskell functions (the functions themselves may be primitives, after all)

page 5, second paragraph: do the parentheses around hPutStr serve more than to confuse? Especially given their absence for hEchoLine...

page 5, footnote 3: comma after effects

page 6: make discarding the return value of forkIO in the second example main more explicit, e.g. with a footnote

page 7, after Atomicity: This ensured -> This ensures

Brecknell 14:32, 11 January 2007 (UTC) I like the way you build up to the definition of "choose" in this new revision. I think this helps to convey how STM and actions as first-class values can improve modularity and composability in concurrent applications. So, if you like, you can consider my lsat comment on the previous revision "done", as well.

Genneth 15:07, 11 January 2007 (UTC) p4: make analogy between () and void to help non-haskellers

ArthurVanLeeuwen 15:10, 11 January 2007 (UTC) The bottom of page 17 has 'Here is his code:' in duplicate.

ArthurVanLeeuwen 15:19, 11 January 2007 (UTC) The note 'The code for choose is brief, but a little mind-bending:' on page 19 is very short, for such a leap in required understanding. In just one single block of 7 lines you introduce both the concept of IO actions encapsulated inside STM actions as well as the concept of list comprehension. For a non-haskeller this is quite a lot.

bottom of p. 4: should "can" be omitted, or should it read "we must explain"?

bottom of p. 7: inconsistent tense: began -> begins.

Should the layout of the do notation be more like normal C layout, to avoid freaking out the mundanes?

would openGate be a clearer name for operateGate?

footnote p. 14: mkGate -> MkGate

santa's code: perhaps a comment between two gates to note that the elves or reindeers do their things at that point.

limitedWithdraw: is the test (amount > 0) redundant?

Fanf 19:45, 12 January 2007 (UTC) Section 4 introduction: The description of the problem reads like a description of an implementation, and I think it should be more story-like. e.g.

Santa spends most of his time asleep. On Christmas Eve, his nine reindeer return from their holidays and wake up Santa. He harnesses each of them to his sleigh,
delivers toys with them and finally unharnesses them, allowing them
to return to their holidays. All year his ten elves are busy working to make toys, but every so often they will need to consult Santa about toy R&D in his study. Santa will only wake up and talk to three of them at a time, and of course their consultations are a lower priority than delivering toys.

p.13 perhaps "sleep" should be "delay" for consistency with the terminology in the code, and to avoid confusion with what santa does - the elves are making toys during the delay, not sleeping.

Also, is it worth noting that the reindeer gates model the harnessing and unharnessing?

Justin Bailey 20:35, 12 January 2007 (UTC) Really interesting paper. I've only been exposed to Haskell (through a graduate Programming Languages course) for 4 or 5 months, but I found it very readable. My comments:

The introduction showing problems with locks and how atomically can be used in the bank account example is very well done.

Development of the gate operations is pretty clear. It would have helped if a short gloss on the connection between the whimsical problem statement and the opening/closing of gates was made. Re-reading, you do cover it in one paragraph on p.13. Sadly, I'd forgotten a lot of that by the time I got to reading about the implementation of Gates and Groups. Maybe make connections throughout that section?

I agree with ArthurVanLeeuwen above about the choose function. It's extremely mind-bending. I didn't event notice actions was a list until I saw the post - I just skimmed past that syntax trying to figure out the do portion. I also think the introduction of foldr1 would throw people not familiar with Haskell or functional programming. I still have trouble with it :)

"The free lunch is over [8]. We have grown used to the idea that our programs will go faster when we buy a next-generation processor, but that time has passed. While that next-generation chip will have more CPUs, each individual CPU will be no faster than the previous year’s model. If we want our program to run faster, we must learn to write parallel programs [9]."

What evidence does this paper offer that using STM will actually get my program to run faster? For all i can see in this example, it may be a beautiful abstraction that when used at scale on interesting programs cannot actually take advantage of the multicore architecture. To be convinced, starting from this particularly motivating opening, i would like to see an example that begins with an algorithm that is not parallel. Then be shown a beautiful, STM-based parallel version that is demonstrably faster.

It would be particularly compelling to see just how a good implementation of STM for Haskell takes advantage of Intel and/or AMD multicore hardware.

It would be even more compelling to see the corresponding lock-based program and how it fairs relative to the STM version in terms of performance, usage of the hardware platform as well as program understandability and analysis.

Clearly, one of the real rubs is getting from current expressions of algorithms to parallel ones, especially parallel ones that map onto modern architectures. Perhaps your point is that STM helps one start to think and conceptualize computation as a concurrent activity -- which then offers some hope to the ordinary programmer to develop programs that will actually take advantage of today and tomorrow's architectures. If so, then the paper is an excellent start, but i would very much like to see this point made more explicit and central, especially if you only give some lipservice to the argument that STM can actually be made to take advantage of the multicore architectures. In particular, evidence for this sort of point might come in the form of a study of how long it takes an ordinary programmer -- not a Simon P-J -- to develop a beautiful solution like a solution to the Santa Clause problem.