Monthly Archive

A little known fact about Canada is that it is the world’s largest producer of famous Americans. Recall, for example, John Kenneth Galbraith, Wayne Gretzky, William Shatner, Michael J. Fox, Malcolm Gladwell, Shirley Tilghman and Keanu Reeves. Some have suggested that Obama is a Canadian, leading to a split in the birther movement between the original birthers and the neo-birthers. Originalists believe that Obama was sired in a Kenyan village, imbued with an anti-colonial mindset, leavened by Saul Alinsky radicalism and smuggled into the US with the intent of turning the US into an Islamic caliphate. The neo-birthers believe that this beggars belief. It is simpler, they say, to believe that Obama is Canadian.

Nate Silver, needs no introduction. While I should have read his book by now, I have not. From my student Kane Sweeney, I learn I should have. Kane, if I may ape Alvin Roth, is a student on the job market paper this year with a nice paper on the design of healthcare exchanges. Given the imminent roll out of these things I would have expected a deluge of market design papers on the subject. Kane’s is the only one I’m aware of. But, I digress (in a good cause).

Returning to Silver, he writes in his book:

One of the most important tests of a forecast — I would argue that it is the single most important one — is called calibration. Out of all the times you said there was a 40 percent chance of rain, how often did rain actually occur? If over the long run, it really did rain about 40 percent of the time, that means your forecasts were well calibrated

Many years ago, Dean Foster and myself wrote a paper called Asymptotic Calibration. In another plug for a student, see this post. An aside to Kevin: the `algebraically tedious’ bit will come back to haunt you! I digress again. Returning to the point I want to make; one interpretation of our paper is that calibration is perhaps not such a good test. This is because, as we show, given sufficient time, anyone can generate probability forecasts that are close to calibrated. We do mean anyone, including those who know nothing about the weather. See Eran Shmaya’s earlier posts on the literature around this.

One of the differences, often commented upon, between economists and computer scientists is the publication culture. Economists publish far fewer and longer papers for journals. Computer scientists publish many, smaller papers for conference proceedings. The journals (the top ones anyway) are heavily refereed while, the top conference proceedings are less so. Economics papers have long introductions that justify the importance of what is to come as well as (usually) carefully laying out the differences between the current paper and what has come before. It is not unusual for some readers to cry: don’t bore us get to the chorus. Computer science papers have short introductions with modest attempts at justifying what is to come. It is not unusual to hear that an economics paper is well written. Rarely, have I heard that of a computer science paper. Economists sometimes sneer at the lack of heft in CS papers, while Computer Scientists refer caustically to the bloat of ECON papers. CS papers are sometimes just wrong, etc. etc.

If one accepts these differences as more than caricature, but true, do they matter? We have two different ways for organizing the incentives for knowledge production. One rewards large contributions written up for journals with exacting (some would say idiosyncratic) standards and tastes. The other rewards the accumulation of many smaller contributions that appear in competitive proceedings that are, perhaps, more `democratic’ in their tastes. Is there any reason to suppose that one produces fewer important advances than the other? In the CS model, for example, ideas, even small ones, are disseminated quickly, publicly and evaluated by the community. Good ideas, even ones that appear in papers with mistakes, are identified and developed rapidly by the `collective’. An example is Broder’s paper on approximating the permanent. On the ECON side, much of this effort is borne by a smaller set of individuals and some of it in private in the sense of folk results and intuitions. Is there a model out there that would shed light on this?