Luck Versus Skill – the Game Designer’s Perspective

Virtually every country has laws which regulate or restrict gambling, because it’s both a fundamental part of human psychology and something which can be extremely problematic if left unchecked. It’s a sticky subject, however, because there are plenty of activities which are fundamental to the functioning of society as we know it which also involve the balancing act of profit versus risk, and we would also not like to discourage more benign forms of competition. Laws therefore have to be worded carefully in order to separate gambling from competitive games and sports, and all of these from more general economic activity.

Unfortunately, the vast majority of the world’s jurisdictions have always opted to put the emphasis on words like “luck,” “chance,” or the equivalents in their own native languages. Wagering on matters of “skill” is generally given greater leeway than similar wagering on “games of chance.” To the lay person, the difference between luck and skill may seem self-evident and clear-cut, so it’s easy to understand why lawmakers would have felt that this hand-waving, blanket approach was sufficient at the time, but there are countless examples of why it is not. The ongoing hair-splitting legal battles in the United States over the difference between sports betting, fantasy sports and poker are only the most recent and consequential example.

I am not a lawyer, so I can’t comment on the interpretation of specific laws, but I have spent most of my life thinking about games and studying their underlying mathematics as well as the subjective way in which players perceive them. What I can say, speaking from that background, is that people’s understanding of the relationship between luck and skill is generally extremely shaky and a massive simplification from the truth.

Two dimensions, not one

If you talk to someone about chance in games, even someone who plays a lot of them, you’ll find that most people see luck and skill as a single axis. They’ll give you something like craps or roulette as the extreme example of a luck-based game, and, at least in Western society, almost always chess as their prototypical game of pure skill. This isn’t wrong, as most gambling games do in fact leave no room for skill, and chess is largely a game of skill. It is, however, a simplistic perspective, because there are many games which lie somewhere in between, and don’t fit neatly on a single axis. How does one compare, say, backgammon to bridge? Or indeed, sports betting, fantasy sports and poker?

In reality, there are two major and largely independent factors which contribute to our intuitive understanding of whether a given game is a game of skill or luck. In other words, what we have is not a linear scale between craps and chess, but a two-dimensional field. Game design is something that people have only started thinking about formally very recently, so there is no fixed terminology, but I’m going to use the terms “luck factor” and “strategic depth.”

Luck factor

Of the two, luck factor is both the easier to understand and, at least on the surface, easier to measure. Put in the simplest possible terms, it describes the likelihood that a clearly inferior opponent can defeat a superior one.

One of the few textbooks written on the subject of game design is Characteristics of Games by Elias, Garfield and Gutschera – Garfield being Richard Garfield, who some readers may know as the designer of Magic: the Gathering. That book brings up “dice chess,” a hypothetical game which I think most game designers come up with at some point while thinking about this topic. The principle of dice chess is simple: It’s identical to regular chess, except that once one player has delivered checkmate, they must roll a die. If they manage to roll a certain result – say 3 or higher – they win, but if they miss, they lose and the checkmated player is considered to have won instead.

Dice chess is pretty clearly a bad game which few people would enjoy more than regular chess. As a thought experiment, though, it cuts to the bone and lays bare what we mean by “luck factor.” In a game of regular chess, a grandmaster can be expected to defeat a beginner 100% of the time, or so close to 100% that it doesn’t matter. With dice chess, however, we could set the probability however we like. If the grandmaster needs to roll 3 or higher on a six-sided die to win following checkmate, then his overall odds are 66.7%, but by adjusting the rules and the dice used, we could make it anything we like, between 50% and 100%. This is, in essence, the game’s luck factor.

What’s important to recognize about luck factor is that it isn’t the whole picture. Dice chess is still effectively chess, and contains as much room for improvement as regular chess, so long as you don’t allow the checkmating player’s odds drop to 50% or below. There’s simply more variance in the end result. But there are plenty of other games, like Yahtzee, whose outcomes hinge on some combination of random chance and decision-making, but which feel more rooted in “luck” than dice chess. How we make that distinction brings in our second axis, “strategic depth.”

Strategic depth

Strategic depth is a much trickier thing to pin down, but has to do with a game’s learning curve and the extent to which a person can productively study it. Here, the go-to example of a low depth game is tic-tac-toe, while again, chess is typically used as the high-depth counterexample.

Tic-tac-toe does not involve any luck at all, except in determining the start player, and in any case, results in a tie under perfect play. It is, however, possible to be better or worse at it. Two perfect players will tie every game, but an imperfect player will sometimes lose to a perfect one, unlike in craps where any two players will perform equally in the long term.

What’s important in looking at tic-tac-toe as an example is that perfect play is within human reach – you can memorize the correct response to every move in an afternoon, if you’re so inclined – and so the skill ceiling is strictly capped. Checkers, meanwhile, has also been solved, yet there is a wide spectrum in human ability, and games like chess still have no practical ceiling for the time being.

Quantifying the strategic depth of a game essentially involves looking for the longest hypothetical chain of players you can construct such that each is measurably better than the last. You have a complete beginner at one end, and someone with full knowledge of the optimal strategy at the other: How many people can you insert between them, each of whom is clearly better than the last?

In the case of tic-tac-toe, there’s only so much to be learned; it’s easy to come up with a few general rules which lead to a win more often than random play, but it’s only another short jump from there to a completely perfect strategy. On the other hand, if you walk into any chess club, you’ll find that all the players have a ranking and know where they stand in the hierarchy, with the only disputes in standing existing between players who are extremely close in skill.

The sum of the parts

It may not be immediately obvious why strategic depth relates to luck factor, but the reality is that the former determines the extent to which the latter matters. Dice chess is still essentially chess; if the checkmating player wins two-thirds of the time, then a grandmaster will beat a novice two-thirds of the time. There may be short-term variance, but the long term winner will depend purely on skill.

Conversely, if you played a variant of tic-tac-toe in which ties were resolved by coin flip, then in a population of experienced players, you’d expect that the actual tic-tac-toe portion of the game would rapidly become irrelevant, because all games would result in ties and be resolved by the flip of the coin.

What’s important here when it comes to sports betting, fantasy sports and poker is that it’s not one aspect or the other which defines whether a game is “mostly luck” or “mostly skill,” but a combination of the two. Dice chess is still chess, but “dice tic-tac-toe” isn’t really very different from just rolling dice. Given that monetary stakes incentivize people to train hard at a game, strategic depth is the more important factor, but it can be hard to discern beneath the more obvious luck factor.

Measurement problems

Unfortunately, it’s hard to quantify either component of a game’s luck and skill aspects.

The luck factor is the easier of the two, but still not as simple as it seems. On the surface, all you have to do for something like dice chess is look at the dice odds, but even games of “pure” skill like chess contain a certain amount of variance when played by humans. If this were not the case, you would expect that every match played between international grandmasters would play out the same way, with the better player winning, but the reality is that everyone commits errors occasionally, and slightly worse players sometimes beat slightly better players even in a game like chess. Thus, on top of the dice odds, we have to include the “blunder factor” of a given game, by which mental lapses – or moments of brilliance – can affect a human player’s performance.

Assuming we’ve established a game’s luck factor, it’s still not obvious how to measure strategic depth. An intuitive and reasonable starting point for establishing skill levels is to take some fraction of the maximum edge. For instance, if a game’s luck factor limits the better player to winning 66.7% of the time (2-1), then perhaps we take a tenth of that edge to define a skill level; you are then one level above me if you can beat me with 1.2-1 odds, or around 55% of the time. If we could do that, then it would just be a matter of taking a population of players and counting the skill levels which exist.

Unfortunately, here we have both the problems with quantifying luck factor in the first place, and an additional problem in establishing a baseline: That is, what constitutes a beginner? The most objective approach would seem to be to assume that a beginner plays completely at random, but of course this isn’t the case. Even a complete beginner at poker is unlikely to fold four of a kind, and a beginner daily fantasy sports bettor isn’t likely to draft a team of all bench-warmers. In almost all cases, some of a game’s heuristics (basic strategies) will be apparent to most players once they understand the rules, so establishing a skill floor can be at least as hard as establishing a skill ceiling.

No easy answers

If you were hoping that I was coming around to a solid opinion on what does or does not constitute a game of skill, I’m afraid that I have to disappoint you. In fact, it’s the extent to which I’ve thought about such things which leads me to throw my hands in the air and say “it’s complicated.”

What I can say is that the complicated reality of the situation makes semantic wrangling pointless. The future of skill-based gambling will ultimately come down to politics, not an objective assessment of various games’ luck components. With sufficient money and energy, it probably would be possible to produce a quantitative comparison of sports betting, fantasy sports and poker, but that probably isn’t going to happen, and it’s certainly not possible to establish through cursory examination because the two components of the luck-skill space have a tendency to mask each other.

My personal opinion is that lawmakers should tend to err on the side of a more, rather than less permissive approach here, because things are so fuzzy, and any strict boundaries will inevitably be pretty arbitrary by the same token. I think it also means that future laws need to be worded very carefully, as words like “chance,” “luck” and “skill” are not nearly as easy to define as we tend to think they are.

Alex Weldon (@benefactumgames) is a freelance writer, game designer and semipro poker player from Montreal, Quebec, Canada.

* WORLD SERIES OF POKER and WSOP are trademarks of Harrah's License Company, LLC ("Harrah's"). Harrah's does not sponsor or endorse, and is not associated or affiliated with PartTimePoker.com or its products, services, or promotions.