29 July 2007

How much is it worth to win at Jeopardy!?

Most weekday evenings at 7pm I watch the television show Jeopardy! One day I'll be on the show, if I actually get around to auditioning and then I get lucky enough to get picked. For now, I just scream "how could you not know that!" from the comfort of my living room. I bet all the contestants do that.

For those of you who aren't familiar with the game, it is a game where three players competing against each other answer trivia questions (although Jeopardy! has this silly trope where your answer has to be "in the form of a question", that's entirely irrelevant here). If they get the questions right, they gain money; if they get them wrong, they lose money. Those players who have a positive amount of money after the first sixty questions (which occur in two rounds of thirty; each round has six categories of five questions, worth varying amounts) get to participate in "Final Jeopardy!". Most players end up with a positive amount of money, since they know themselves well enough to only attempt to answer questions which they think they know the correct answer to. The players are told the category of the question; they then can wager any or all of their money that they'll get it right. Then the question is stated and they have thirty seconds to write down the answer.

Fans of the show have put together a Jeopardy! archive. There's a wagering calculator available online that takes into account common wagering strategies. But this does not take into account the following facts:

only the player who wins the game gets to keep their money (probably an average of $20,000 or so); the second- and third-place players get $2,500 and $1,000 respectively. (Incidentally, the show's host, Alex Trebek, seems to refer to in-game totals as "points", which I suspect is tied to this;

perhaps wagering strategy should depend on how likely one thinks one is to get the question right, and how likely one thinks one's opponents are

the player who wins get to come back the next day; the others don't.

The first two here, I plan to address in a later post. It's the third one I want to talk about right now.

So first we must answer the question -- what's the probability of winning one's second game, given that you've won the first? Of winning one's third game, given that you've won the first two? It's obvious that a defending champion is probably better than the average player (because they've won at least one game), but how much better?

Fortunately, the Jeopardy! Archive can help us answer that. It would be enough to know what proportion of champions win one game, two games, three games, and so on. The archive compiles an impressive set of statistics for each season, but this is not one of them, so I have to do it myself. There's a natural cutoff point in the data -- for a long time Jeopardy forced its champions to leave after five games, but they don't do that any more, since the beginning of the 2003-04. (This led to Ken Jennings' historic 75-game run in 2004.) I originally planned to go back to the beginning of that season, but they only go as far back as the beginning of the Ken Jennings run, near the end of that season.

Now, if there was no effect like the one I just mentioned, you'd expect there to be three times as many one-game winners as two-game winners, two-game winners as three-game winners, and so on. It actually seems like if there is such an effect, it's not that strong. I attribute the big peak at five games to psychology; it's probably hard to win a sixth game because you're going into some sort of "unknown" territory. Notice that only four players -- Ken Jennings, David Madden, Tom Kavanaugh, and Kevin Marshall -- have won their sixth game.

So I'll assume that there is no "memory effect" -- that if you win today, you have a one-in-three chance of winning tomorrow. This seems believable -- the categories can be very different from day to day -- but I've never seen this analysis before. (It wouldn't surprise me if other Jeopardy! hopefuls have done it, though, because they seem to be That Sort Of People.)

Thus, when wagering in Final Jeopardy, one should wager as if the prize is not just the money you're going to win -- but one and a half times that much, since you can expect to win half a time more. The average champion is a one-and-a-half game winner.

But there are two problems:- how do you use that information? Does the amount of money one expects to win really affect proper wagering strategy?- more importantly, you only get to play at Jeopardy! once. I think the rules say that; in any case, I've never heard of somebody who's played twice on the Alex Trebek version of the show. (There are a few cases of people who played on the Alex Trebek version and on some prior version.) So anything you say about "expected value" is meaningless! What's the point of an operation that talks about the average amount you expect to win if you don't get to play long enough for that average to take effect?

8 comments:

[Y]ou only get to play at Jeopardy! once.... So anything you say about "expected value" is meaningless!

Okay, I must preface this with the disclaimer "IANAP/S".

I'll flip an unbiased coin with you to trade my salary for your stipend for next year. But I'll only flip it once. So the "expected value" of accepting this wager is meaningless, according to your reasoning, right?

I suspect that people who apply expected value analysis to their decisions, even those decisions which are not themselves repeatable, come out ahead on their decision-making over time by a fair amount.

I agree. For example, Nassim Taleb (author of Fooled by Randomness and The Black Swan, and from what I understand a very successful trader) has publicly said that his strategy consists of making many such decisions. On most such decisions he (or his firm) loses money, but on a few such decisions he gains money and the net return is positive. I was thinking from the point of view of the return entirely within the Jeopardy universe; obviously the people on the game show exist outside of it as well.

I am curious as to why (or in what ways) you believe the J! Archive wagering calculator does not take into account those three bulleted criteria that you mentioned. For wagering purposes, all of them boil down to "you ought to wager for the best possible outcome for which you are positioned". The rest of your post has no bearing on wagering strategy.

Naturally, those who have very high Final Jeopardy! conversion rates should tend to wager toward the martian end of the suggested wagering ranges rather than the venusian ranges. (Ken Jennings, with his 67% FJ! conversion rate, lost a sizable chunk of change by wagering more or less venusian over his career.) But even a player with a high FJ! conversion history ignores the 2/3rds rule at his peril, because FJ! get rates among the three players responding to the same clue show good correlation.

Since 2002 "Wagering-Sports.com" has aimed to make sports more enjoyable. Whether you just want to know the odds or want to get our analysis of the game, we want to be your first source on the internet.

We would really appreciate if you can give your suggestions/feedback/reviews for improvement.