Loot, Damned Loot, and Statistics

There really wasn’t much paladin-specific information provided in the MoP media event – at least, not the sort of crunchy, math-y sort of information I usually blog about. However, there was a lot of talk about the new LFR loot system they’re going to put in place, and a lot of controversy/outcry/confusion on the part of the players. I worked out a little bit of the math during an e-mail conversation with Anafielle, and we decided it was interesting enough to turn into a blog post after all.

I think the system is good, it just wasn’t explained very well at first. And there’s still some confusion, even on Blizzard’s part it seems, about whether your roll is truly independent of other players or not. But for the moment, let’s assume it is (either system works, really – I’ll touch on that again at the end).

The most common complaint I’ve seen in the past few days is that players feel like they’ll have less control over what they win. “But if item X drops,” they say, “I only have to roll against the other people that need it, and not the people who don’t need it.” The flaw in this logic is that “if item X drops” encapsulates a whole bunch of RNG that isn’t in their favor. That statement self-selects a small subset of the possible scenarios (e.g., that item X has dropped) and compares that subset to the entirety of the new system, which isn’t a fair comparison.

In other words, you don’t have control in LFR, you have the illusion of control. If it were a guild run, and you could guarantee that “next time item X drops, it’s my turn,” then you’d have control. If you’re alone in an LFR, the expectation is that all 24 other people in the raid are after gear, and all of them that can roll on your item will. Unless you’re participating in trading cabals, you don’t have any more control over what drops and whether or not you win it than you do in a completely random system.

To show this, let’s compare the systems. The correct way to do that is as follows:

Current system:

chance_to_win_X = (chance_X_drops) / (num_players_rolling_on_X)

Consider that most items have a ~15-35% drop rate, so let’s use 25% as a baseline for item X to drop from the boss. Note that this is after all of the combinatorics involved in the drop happening (i.e., did it drop once, did multiples drop, etc.). Let’s say that 4 other players in the raid are rolling on the item as well. That means you’ll have a one in five chance (20%) of winning the item if it does drop.

Your net chance of winning a specific item in the current system is (0.25)/5 = 0.05 = 5%. And it gets worse – far worse – if more people roll. Have 8 people roll and you’re down to around 3%. Have a Vanquisher token (10 rollers in an average 25-man group) and it’s down to 2.5%.

Your net chance of winning any item (i.e. not just item X, but item Y or Z from the same boss as well) is a lot more complicated, because it depends on several details about the boss’s loot table. How many items does he drop? How many of those items can you actually use? What is the individual probability of each drop (i.e. on a tier table, Vanquisher is 40% but Conqueror/Protector are each 30%)? And it’s complicated further by the fact that the loot table is usually split in two to prevent 4 of the same item from dropping. So for example, if the boss drops 4 items, 2 are from one loot table and 2 are from another. Bosses that drop tier generally have one table for the 3 tier tokens, and one table for non-tier drops.

So let’s make some assumptions. We’ll say the boss can only drop 2 different items you can use, which seems reasonable for DS after quickly browsing a few drop tables on wowhead. Let’s also assume those two items are on separate tables – maybe one is a tier token, and the other is an off-set item like a ring, bracers, etc. That makes the two item rolls independent, which means your chance of winning either or both of the two items is (1-(1-0.05)^2) = 0.0975 = 9.75%.

That might look confusing, but it’s actually quite simple. You have a 5% chance to win one drop, and a 95% (1-0.05=0.95) chance to not win that one drop. The chance of winning one, the other, or both drops is just 100% minus the chance that you lose both drops (because there are only four possible outcomes: win+win, win+lose, lose+win, and lose+lose). The chance that you lose both drops is 0.95^2, so the chance that you win anything is 1-0.95^2 = 0.0975 = 9.75%.

New LFR system:

chance_to_win_X = (chance_you_roll_high_enough) / (num_usable_drops)

Let’s say the boss drops 2 items you can use, and assume they’re equally probable. Let’s also say that Blizzard sets the roll threshold at 90 (i.e., you have to roll 91 or higher to win an item – a 10% chance to win). You have a (0.1)/2 = 0.05 = 5% chance of winning the exact item you want. But you have a 10% chance of winning at least one item that you need.

I’ve simplified things a bit here by assuming all items are equally probable – in other words, if there are 3 usable items the boss drops, each has a 33.3% chance (repeating, of course) to be the one you get. That isn’t necessarily the case. They could have some items be more common than others by weighting things differently (i.e. tier token is 40% chance, each of two off-set items is 30%). But that doesn’t really change much, apart from making specific items more or less likely. It doesn’t change the average.

Also note that we’ve made up a lot of these numbers – they could make it a 15% chance to get any item (increasing your odds for both overall and for single items), and they could distribute loot such that there’s 3 or more items you’ll want off of a given boss (decreasing your odds for a single item – but this becomes less likely in a tier with more bosses, it was much worse in Firelands and Dragon Soul than it was in T11 or will be in T14).

Declaration of Independence

I said in the beginning that it doesn’t matter whether your roll is truly independent of other players. Here’s why:

In the new system, your chance of winning is the chance you roll high enough divided by the number of useful items that boss drops for your class. Now, they could set “chance_of_rolling_high_enough” to an arbitrary value, like 10%. That’s what I did in my example. Then you’re truly independent of other players, because you could all win one item, or nobody could, or anything in-between.

What happens if instead, the system performs a /roll for each player and just awards items to the top 4? Well, it means that instead of rolling to beat a certain arbitrary threshold, you’re rolling to beat 21 other players. In other words, you need to be in the top 4 out of 25, or in the top 4/25 = 0.16 = 16%. It doesn’t matter who those players are, either, because they’re all rolling automatically. So if they did implement that system, it would be no different on a personal level than just setting the bar arbitrarily at 84 in the independent system. You’re not really competing against the other players for anything, because statistically it will always work out such that you’ll have a 4 in 25 chance to win once you average over many boss kills. The only difference is that the variance in the number of items awarded goes to zero. In other words, there’s always 4 items awarded rather than randomly fluctuating between 0 and 25 with an average of 4.

Conclusions

Until we know the percentages, we won’t know for sure which system truly gives you faster loot accrual. But it’s far more likely that your chance of winning items will increase with the new system. And more importantly, it will be much more consistent, because it won’t be as subject to group composition. No more zoning into LFR and groaning because there are 5 DPS DKS, 3 Ret paladins, and 4 Warriors all trying to get a two-hander.

The “extra bonus roll” stuff looks interesting too – it’s basically a way to weight the dice in your favor for a particular boss, which has some neat strategic implications. If you’re really after item X, you can load your dice for that particular item. Sort of nifty.

Given that the new system streamlines the looting process, eliminates loot drama from LFR, and adds the ability to use loaded dice on certain bosses (something that would be abysmal to implement in the old system), I’m on-board. Especially since the only cost is the illusion of control.

18 Responses to Loot, Damned Loot, and Statistics

So I read this, and it is interesting and pretty much the same math I came up with. But…

At the end of the article is a video with Steve Nash, and I was scratching my head trying to figure out how you were going to tie Steve Nash into the system. Imagine my disappointment when I realized it was just an advertisement for men’s soap.

I hate this system, instead of a strained analogy to Steve Nash all I got was soap.

I reckon if they want to avoid numerous frivolous tickets, they will HAVE to implement the top 4 /roll’s win a prize over the 25 individual 16% chance rolls. After all if you are in a raid an NO ONE wins any loot, there WILL be QQ… or even worse, 24 people win a prize, but YOU don’t.

I could see it going either way. The advantage of the “true independent” system is that you really cannot complain about the other players – your result is totally independent of who and what they are.

In the “top 4 /rolls” system, there’s still that implicit level of competition, even if the rolls aren’t public and the system is statistically identical on long time scales. People will still feel like they “lost” to the four winners.

But in the end, since you *cannot* choose to forego rolling (i.e. pass), the two systems are functionally identical for large sample sizes. I suspect they’ll go with the “top 4″ idea just because I agree it would be weird to see 15+ people win an item on one boss.

Using your example of a 90 roll threshold and 2 items you could use, there’s a 5% chance of each item, and a 10% chance of you winning an item you need. However, once you’ve won one of the items, the next time you win the roll, there’s still only a 5% chance of getting the second reward (since you are equally likely to get a repeat). That means you’re looking at an average of ~30 kills to get the two items.

God forbid there are three items you could use. Let’s ignore the odds of actually rolling a winning roll, and just look at what happens when you do: There’s only a 2/9 chance that after 3 winning rolls, you’ll have all 3 rewards.
There’s a 33% chance you’ll still be missing something after 4 wins, a 23.5% chance after 5 wins, 16% after 6 wins, 11% after 7, etc… It comes out to just shy of 5 wins on average to get your 3 items. For the most unlucky 1% of people, you’d still be missing an item after *13* winning rolls.

Then you should factor in the odds of winning. I’ll assume a new tier of content comes out every 6 months (26 raid weeks), and you’re doing LFR every week.
If you use extra generous numbers that say a roll of 67 or higher wins an item, about 5.5% of people wouldn’t have all 3 items by the end of the tier.
At a more realistic “5 items per boss” rate of needing a 81+ roll to get an item, ~21% of people would still be waiting for an upgrade by the end of the tier.

And that’s just from 1 boss.

I’m not sure how that compares to the current LFR system, where you have to have the server “roll” your loot, and then beat out your competition, but it worries me. I’d rather lose the roll on an upgrade a few times than never even see it, while I have multiples of the other drops.

In the current system, if two rogues are in the group, Rogue 1 needs item A, and Rogue 2 needs item B, and both items drop, they can swap and both walk away winners. In the proposed system, if both rogues have a winning “roll”, there’s only a 25% chance that they’ll both get what they need. This is a flawed comparison, but it illustrates my point that the potential for loot rot is greater when you can’t exchange rewards, and need every person to win their loot lottery individually.

Just to quickly add – SWTOR’s PvP loot system was similar to the proposed WoW system before it was patched 2 months in. You had 13 item slots, maybe a 20% chance of a token, and the token could be for any slot. While you got your first few upgrades pretty quickly, you started getting doubles and triples very quickly, and it took longer to get your last few items than it did to get the first 10. It was the most frustrating loot system I’ve ever encountered, and I ranked up for HWL gear in Classic WoW.

As far as how it compares to the current system, it’s almost no different, with certain caveats. What you’re describing is statistical independence – namely, that the outcome of a random event doesn’t depend on previous results.

Now, that’s not technically fair for LFR, for exactly the reason shown in your example. You might get lucky and find that one or more other players in your group have the item that drops, and you end up having a higher chance to win as a result. That introduces an element of interdependence between previous kills and the current kill.

However, I feel pretty confident saying that’s going to be lost in the noise, on average. There are also people who will roll on gear they already have so they can trade, and others who will roll on it just to *vendor* it. Those of us that actually pass on stuff we already have, especially tier tokens, are in the minority.

So in general, LFR is pretty close to the ideal statistically-independent Bernoulli trial. You’re paired with around 24 other people, all of whom are going to be rolling on any gear they can use.

And the case of the poor 1% that win 13 rolls in a row but still don’t get the item they want has its corollary in the current system. The item may not drop, or it may drop and they get out-rolled multiple times in a row. Those sorts of streaks do happen, and yeah, they suck. But it’s not a new phenomenon, nor is it unique to either system.

Actually, the players that really lose out in the new system are Holy paladins. Because there’s generally only going to be one or two in an LFR, giving them a higher chance of winning any intellect plate that drops than, say, a rogue does on leather gear or a Ret does on strength plate. In theory, the drop tables should drop INT plate less often to compensate for this, but it doesn’t seem to do so proportionally (i.e., 5% compared to 30% for other items – it’s usually still a ~15% drop rate minimum).

So, yeah, Holy paladins get a bit screwed by the new system, and Ret/Arms/Fury/Frost/Unholy benefit, as do casters.

Maybe this is the career guildie in me, but I’m completely happy with the system with a single exception: I’d like the option to give up my chance at loot. Granted, I may be in the minority here, but I pass on stuff in LFR even if I don’t have what dropped provided it’s a better upgrade for someone else. (For example, The stam with mastery proc trinket I’ve passed to other tanks after I ask if they have it.)

Once I have everything I want from a boss, I’d want to be able to pass and give the others in the group a slightly better chance at loot (assuming the system gives loot to the top X players based on a roll).

So yeah, give me that one option and this will have my full support rather than the near-full support it has now.

You’re mixing things up and adding different concepts here. Blizzard haven’t yet decided how to handle the loot. It’s either going to be “Each player has X% chance to win” or “Boss drops Y items and Y players win”.

In the first case, your odds are exactly X. In the second case, your odds are Y/N where N is the number of players in the group. You’ve combined the two scenarios here in your analysis and that’s not one of the options they’re exploring (to my knowledge)

No, I’m not. I addressed the two concepts separately. The first is the “X% chance to win,” after which it chooses one of the items usable by your class. The second is “Boss awards Y players loot,” which is again chosen based on your class and spec.

The first system was described in the interview with Cory Stockton, which was linked in the first paragraph of the blog post:

“So the biggest change is that we’re moving towards an individual, per-player loot system. Quite similar to how loot works in Diablo 3, roll loot for each player and you have a chance. Now, this doesn’t mean everyone’s going to get a roll each time–you’ll win a roll, but that doesn’t necessarily mean you’ll win an item. You might just get gold instead. But I think it will hugely impact the feeling of LFR.”

The second system was described in the interview with Greg Street, which was linked in the second paragraph of the blog post:

“So a couple of things we’re doing that are a little different this time is, in Raid Finder and for the world bosses, we are not having loot rolls. We’re having per-player loot. So what this means is, the game will decide that X number of players, say it’s 3 to 6 in a raid, are going to get loot this time, and then it gives them loot that’s appropriate for their current spec, whether that’s their main spec or offspec. “

I do find it a bit disappointing that Blizzard seem to be removing the possibility of gearing your offspec while not playing it. I play almost all the time as a tank, but I have acquired some healing loot over time (only pally in our raid so I get the int plate by default despite any protestation), I recently ran through LFR a few times with the intention of gearing up my last few slots, but it’s virtually impossible to get anything. Since my tank gear can’t really benefit from anything in LFR I simply don’t run it any more.

I understand the reasoning in the current system, as it would suck to lose your main spec gear to someone just gearing their off spec, but in the new system it would be great if you could nominate the tree you would like to receive loot for.

You do lose that flexibility in the new system, but I’m not sure how big a loss that is. I’ve rarely seen anyone win something for off-spec in LFR, simply because there are usually main specs that get the roll bonus. Paladins rolling on Holy off-spec gear are probably the only ones with a low enough spec representation to get a significant amount of loot that way. Tier tokens are one of the few things that most specs can get for off-spec via a main-spec roll, and that should still be possible in the new system if they keep the token implementation.

That said, if you want Holy gear, queue as Holy! At least healer queues for LFR are instant. I’ve been queuing as Ret (~20 minute queues) for the last few weeks to try and pick up off-spec gear.

You don’t have to work out the formulas to prove your point, though it’s interesting to do so. That would be a good math-class exercise compared to drawing marbles out of a bag! The short version, though, is that any formula is going to involve constants, and Blizzard can always fine-tune the way the system works by adjusting those constants.

What’s better about individual rolls is the effect it has on the social dynamic. No more kicking of marginal players because they share loot with you. No more stacking an LFR raid with guildies to give your guys an unfair advantage. No more after-drop discussions about trading the loot around; while those add a good element to guild runs, I have found them to be buzz kill in LFR runs.

LFR is about strangers getting together, for an hour or two, to kill some Internet dragons. Because an LFR raid lacks the ongoing social context of a guild, it needs a simplified, fast-paced, and built-in loot system to a larger degree than guild runs. I’m glad about the switch to individual rolls. I don’t care if it’s a “high enough” roll or a “top 4/25″ roll. It just needs to be independent of who else is in the raid, and it needs to forbid trading of dropped items.

You bring up a good point, but one that brings up a question. Will we still see top end guilds planning out their LFR runs for maximum loot-efficiency?

Barring those that exploited, the high end guilds that did game the LFR system did so by running 22 saved people through an LFR run along with one player of each token type to maximize loot distribution to them, with a particular emphasis on tier tokens.

The “X players per boss get loot” method will see this continue with a renewed fervor because then they’ll know that by running 25-X saved players through along with the X unsaved players, they can maximize loot and virtually guarantee that something good will come out of it because it won’t all be caster mail (a common affliction for my guild).

If they instead go with the variable system of “each player has a Y% chance of getting loot” method, we’d see this sort of farming stop completely.