Average Your Guesses

What percentage of the world’s airports are in the USA? (Answer below the fold.)

Take a guess. Now, take another guess, different from the first one, and average them. According to research reported on in The Economist, averaging the 2nd guess improves accuracy by 6.5%. Better still, wait 3 weeks before taking a second guess. Averaging now improves accuracy by 16%. (Story found via Slashdot.)

It is important that neither group knew they would be required to furnish a second guess, as this precluded subjects from misinterpreting their task as being to specify the two endpoints of a range.

That could make it a little tricky to do this on your own; you have to try to make your first guess as good as you can, and then start fresh for the second guess.

This benefit of averaging cannot be attributed to subjects’ finding more information between guesses, because second guesses were less accurate than first guesses

Hmmm, how can second guesses be less accurate than first guesses, yet averaging them is more accurate than either? I suppose it means that the two guesses tend (on average) to bracket the correct answer, with the second guess farther away than the first one. That means that your first instinct to improve your guess is more likely than not to be in the correct direction, but go too far. Knowing this might allow you to improve your guesses even more.

Oh, and as for the airports? According to the CIA World Factbook, there are 14,947 airports in the U.S., and 49,024 in the whole world, so 30% of the world’s airports are in the U.S. For the record, I guessed 25%, and then 15%, so averaging didn’t help me. But in general this might be a useful trick to easily improve guesses.

I guessed 20% (first number that came into my head) and 25% (USA’s share of world economy).

Incidently the average of everyone’s guesses so far is 25.3%.

Divide

Hey, this doesn’t seem to work! I guessed 30% the first time. (With 10% being the second guess.)

Tiiba

I remembered the US having about 25% of the world’s GDP (vague memory), and considered that airports are a luxury, so rich countries probably have more, relative to GDP. So I thought 1/3, which seems close enough. By the time I realized I need a second guess, it was too late. Can we play again?

I guessed 50% and 1/6th, based on half-remembered ideas about natural resource usage, economy size, etc. Final result: 33%, almost exactly on the mark.

Anonymous Coward

Hah. I guess I’m one of the ones for whom the averaging helped – I guessed 40 and 20, to end up with 30 on average. I didn’t have too much reasoning behind the guesses though. I thought the US would have a lot more than their share of airports, so I guessed 40, which seemed like a lot; I thought about it a bit more and figured I should adjust that guess downward, it seemed high, so I guessed 20.

I could definitely have been biased by the fact that I would have to make two guesses, since I saw that sentence as I was thinking about what guess to make. This entry really isn’t that great of a test because of that. But hey, it was never really meant to be.

This fits with the idea of temporary mental noise – even if there is more mental noise in the second estimate it is better to average the two estimates to reduce the noise.

James Andrix

I also used the 25% of the wealth number for my first guess.
Then I visualized Major cities around the world (‘earth at night’ type images, mainly) but completely failed to think about minor airfeilds. So I guessed 50. Average 37.5

Second guess was sloppy, so I lost 2.5%.

On the other hand, My first guess might have only been so close because it I had a roughly related statistic to go on. if I had only remembered my other vague number about the US being 6% of the population, I might have started with that and increased it a bit to account for wealth. If I doubled it, and still had the same second guess I’d be cheering this method.

So maybe a topic that is too well-known yeilds a first guess that is too good for this method to work reliably.

Si

I’ll have to think twice before buying another lottery ticket.

Grant

I wonder, does this second later guess say anything about the common folk-wisdom to “sleep on” important decisions?

@”I wonder, does this second later guess say anything about the common folk-wisdom to “sleep on” important decisions?”

It suggests to me that you can’t ever “start fresh for the second guess” as Hal suggested you should.

Daniel Wolfe

This reminds me of a novel by John Brunner, called The Shockwave Rider. In the novel, there exist these groups called Delphi Pools, which consist of a group of normal people who all make guesses as to what the turnout of something is going to be. Surprise surprise, their averaged guesstimate is actually pretty close to the mark.

So, I know it’s been used in fiction before.

michael vassar

First guess 25%, based on fraction of world economy 10-20 years ago, second guess, based on the observation that poor countries have limited highway infrastructure and need more very small airports, was 10%. I suspect that the second guess is more accurate for sufficiently small airports despite what the article says. I’d give even odds on the USSR having more airports than the US for light craft and the remainder of the world more than either alone, making the world factbook number impossible. I wonder what definitions they use.
Sorry Robin and Hal but experience teaches me to be something that you probably consider overconfident.

Nick Tarleton

“Airports: This entry gives the total number of airports or airfields recognizable from the air. The runway(s) may be paved (concrete or asphalt surfaces) or unpaved (grass, earth, sand, or gravel surfaces) but may include closed or abandoned installations. Airports or airfields that are no longer recognizable (overgrown, no facilities, etc.) are not included. Note that not all airports have accommodations for refueling, maintenance, or air traffic control.” – Notes and Definitions

Just like Hal Finney, I guessed 25% the first time and 15% the second time. The phenomenology of second-guessing is quite interesting. Updating your original guess in one direction feels right at one moment, and the next moment what feels right is an update in the opposite direction. I moved back and forth between 15% and 35%, and only settled for the former figure because I was impatient to see the right response.

Apep

Yes, I thought my first guess was too high and I guessed higher the second time because that was what came to mind first. The mental sequence went more or less like this: “25%. Hmmm. That seems a little high. 35%. So maybe 15% is a better guess.”

I guessed 25%, but I had an unfair advantage – I designed NASA’s airspace simulator, and had a list of all the international airports in the world. I believe I had just under 20,000 US airports in my simulation. That would include all airports for which ETMS data is filed, but would exclude a large number of private & small-plane airports.

Daniel Wolfe, Gwern: The Delphi method is different from but related to Delphi pools, which (IIRC) were initially just averages of guesses by a set of people but very soon incorporated betting on outcomes, essentially becoming prediction markets ( http://en.wikipedia.org/wiki/Prediction_market ), a topic Robin Hanson has written a lot about. See http://hanson.gmu.edu/ and search OB for those.

1. The Market and Human Nature — from a 2005 QA with the conservative scholar Roger Scruton (via Rod Dreher): MG: What deleterious consequences result from the “free market ideology” you mention? Are there particular economic arrangements that co…

Just a note: the paper does not claim that the average is always better than either guess! Only that you should expect it to be so. Of note, second guesses were worse than first guesses (overall, not always), but it was still better to average (overall, not always).

Phillip Huggan

This effect is because you tend to guess in regular interval jumps. Like fives or tens. So your first guess is closest, and the next is 10% away in the right direction. Later guesses might tap unconscious brain work. 20% and 25% for me.