Friday, September 13, 2013

Markov Chain Monte Carlo Seminar

This semester my colleague Peter Beerli is running a seminar in a journal club format. We are trying to read the most seminal papers in MCMC.

Of course, one has to start at the beginning and read "the" famous paper by Metropolis et al., "Equation of state calculations by fast computing machines" which has been cited more than 25,000 times. It had been a while since I had read the paper (more than a decade ago, I think), and I enjoyed re-reading it after the hiatus.

It was like re-reading one of those books you read as a teenager, and taking completely new things away from it. When I first read it, I was caught up with the mechanics, since they were still very new to me. This time around, I got to marvel at some of their astute observations (and lament their choice of a horrible random number generator).

I also spent some time reading the background of the paper (some links on Peter's webpage). In a relatively recent interview, one of the paper's authors Marshall Rosenbluth had this to say about some of the other co-authors:

Barth: Your collaborators for these papers were Edward Teller and Nick Metropolis and your former wife?Rosenbluth: Yes. She actually did all the coding, which at that time was a new art for these new machines. You know, no compilers or anything like that.Barth: And it’s also listing A.H. Teller.Rosenbluth: That was Teller’s wife, who during the war had been one of these computer [women] - he wanted her to get back into the work, but she never showed up. So she was basically -Barth: Put on the paper for it?Rosenbluth: Yes. As was Metropolis, I should say. Metropolis was boss of the computer laboratory. We never had a single scientific discussion with him.

It is very interesting that none of the coauthors made any subsequent use of the algorithm.

The last bit from that paper:

Obviously, Marshall’s revelations prompted hallway and diner discussions about whether the Metropolis algorithm should be called the Rosenbluth or at least the Rosenbluth-Teller algorithm. One of the organizing committee, Rajan Gupta, whose curiosity got the best of him, privately asked Marshall his opinion about the name of the algorithm. Marshall replied:

“… life has been good to me. I feel rewarded in knowing that this algorithm will allow scientists to solve problems ranging from fluid flow to social dynamics to elucidating the nature of elementary particles.” The original name will stick.

Three times during the conference, Marshall walked up to me and said, “It hard to explain how exciting it was to be at Los Alamos during those times. To be able to interact with Teller and von Neumann was very important to me.” Even heroes have their heroes.