Thursday, July 16, 2015

Your Stat Weights Don't Matter

The Grumpy Elf recently made a post called "Don't Judge a Character By Their Item Level." His general argument is that item level is a terrible way to judge a character's potential performance because things like stat weights and set bonuses can wildly skew results. An example of this that he mentions is using four 670 tier pieces from BRF for four set over four 695 items. If I was to form a "thesis" of sorts, this would seem to be it (please note that isn't a single contiguous quote, I cut out some stuff in the middle to focus on the essentials):

I mentioned my 687 item level, which for a raider is pretty poor, but once again that does not mean I would do poorly. Put me along side someone that just did the T2 grind, upgraded anything
they could get, and purchased a few BoEs off the auction house and you
will see someone with an item level over 700 but someone that is not
even remotely capable of matching my hunters numbers. Why is that?

Because the stats on gear matters, the bonuses on gear matters, almost
just as much as hitting the right keys in the right order.

I'm here to tell you that he's wrong. Now before you grab your torches and pitchforks, kindly let me explain.

For those of you who aren't familiar with my history, I started playing WoW shortly after Vanilla launch. It was the first MMO I played and the second game I had played online (the first being Warcraft 3). I was generally quite the "n00b" at the time about, well, almost everything and I didn't seriously start raiding until Burning Crusade. Now, of course, I'm the guild/raid leader of what is currently the top two night a week Mythic guild in the US. There was an incident, however, that still sticks in my mind from those Vanilla days.

My main at the time was a 60 Night Elf rogue who had somewhat recently hit level 60. I was using a level 52ish blue dagger in my main hand and a level 55 green sword in my off-hand since I hadn't had the chance to do many max level dungeons yet. A friend on the server said his guild (which was fairly casual and had recently started raiding 20 mans) were short people for Zul'Gurub (a 20 man raid at the time) and asked if I could go. I said sure. When I got there I saw that nearly everyone was massively better geared than my poor rogue -- full level 58+ blue gear and a smattering of epics (hell, at that point I had never even gotten an epic on any character). There was even another rogue with level 60 epic daggers. To say I was a bit nervous about looking like the worst player in the history of the game would be an understatement. So what happened?

I *crushed* everyone on the damage meters for the two bosses we did (bat and snake bosses). Like, top DPS by 20% and the next highest player was my friend on a Fury Warrior. To say that everyone else was bewildered (especially that other rogue) would be another understatement. But why was I able to do that?

1, I was playing Combat Daggers (yes, Combat Daggers wanted two daggers, I was using an off-hand sword, I didn't have another dagger handy yet that was reasonable, yes it was terrible) while that other rogue was playing Seal Fate (mostly Assassination spec with some Subtlety talents). Seal Fate was insane for generating quick combo points and doing massive burst damage in PvP. It was terrible for sustained boss damage in PvE.

2, I played my spec correctly. This means I did stuff like actually use
my combo points on Slice and Dice (long term attack speed increase)
rather than Eviscerate (instant burst of damage -- Rupture wasn't even
an option due to only having eight debuff slots on a boss back then).
While that might seem like a "Duh!" thing to you now, it was not common
knowledge back then at all.

3, I maintained high target uptime. That means I was careful and quick about my movement so that I could be attacking the boss as much as possible despite either myself or the boss needing to reposition at times. This is sort of similar to the ABC rule (Always Be Casting) of a ranged character if you've never experienced the joy of a melee character.

In short, having the correct spec, playing said spec correctly, and playing my character well resulted in the other rogue switching to Combat Daggers for the very next raid (where I still beat him, but by a much smaller margin).Playing your character correctly is far, far, far more important than optimizing your gear.

But if that's true, then why do we have so many tools designed to give us stat weights? Why do we have stat rankings, gear lists, and so on?

Because those tools are meant for Mythic raiders and, to a lesser extent, Heroic raiders. Mythic raiders, generally speaking, have *already* mostly mastered their characters. There's always room for improvement, of course, but you run into significant diminishing returns. Figuring out how to go from 50% to 80% of optimal DPS is fairly easy. Learning how to go from 80% to 90% of optimal DPS is a harder. Managing to go from 90% to 95% of optimal DPS is harder still. And so on. Then, of course, you need to learn how to optimize for specific fights but that's difficult to practice outside of actually raiding.

At the point where it becomes very difficult to figure out how to squeeze out a few percent more DPS out of your rotation and you're generally comfortable with dealing with raid mechanics, *then* obsessing a bit over optimizing gear starts to make sense. Spending a few hours figuring out the best gear to grab that gives a 2% DPS boost winds up being a better choice than trying to spending that time learning how to play your character slightly better.

But how many players are at that level? Only a few percent overall (mostly Mythic raiders and the "upper" Heroic raiders). Yes, you have the occasional player who's amazing but "retired" and raiding Normals with friends...but that's really a Mythic/Heroic raider in a Normal guild. So what about Grumpy? Well, let's assume for the sake of argument that Grumpy knows how to play his class perfectly and instead focus on his raid group -- Grumpy leads a group trying to work on Normals.

What do we know about said raid group? I'll quote some of the statements he's made in the last few days about it (all are taken from here):

We wiped a few times, all to stupid stuff that is completely avoidable.
I was the middle of the chain, the mage I was attached to came to me,
the other person did not. Even after yelling his name on voice chat to
come to me. Guess who that other person was? You got it, him.

He dropped doom right on melee 2 times. He did not run to the person he
was chained to three times. He did not switch off the boss when I said
multiple times adds were top priority. If there was a mechanic in this
fight, he was messing it up. About the only thing good I can say was
that at least he was consistent, but when that consistent is
consistently bad, that is a problem.
---------------Lets put it this way to show you what some of these people do that I deal with.

When
explaining that fight I put a square marker for where to stand when
phase 2 comes. Someone screws up and stands in melee and drops doom
right on that marker. Phase 2 comes, I say on vent, stand to the left
of the marker (because the doom is there of course right. Common sense
to any raider that has ever raided if you ask me) but what does half the
raid do. They F'N stand on the marker and die in the doom. When I say
something on vent after the answers I get. But you told us to stand on
blue in phase two. aaaahhhhhhhhhh

Sorry for ranting, and no I am not making that up, it happened, it really happened.---------------I will not move the marker. I
am trying to train these people to be better raiders, doing that teaches
them nothing. If they can not adjust on the fly and understand simple
theory such as "markers are not absolutes" and be able to adjust they
have no reason even stepping into a raid.

I need to teach these people. I
will be raiding with them every week. I can not just always move the
marker. That makes for piss poor raiders that do not know how to do
anything but follow orders. They do not understand the mechanic on
their own, they do not become capable of making snap decisions and they
will never learn how to listen to changes on the fly from the raid
leader.
----------------- I have the permanent rune now, so
I do not worry about it and I suggest to all my raiders that they
should get it too. It is only 5K and you never need to buy one again.
Sell the ones you get and you will make more than the 5K you spent on it
back. Casual or not, there is no excuse for not being the best you
can.-----------------As for my guild. DPS is never an
issue. Mechanics are. It takes them a bit longer than I would like to
pick up mechanics than I would like it too. 1% more however is 1%
better. The faster the boss goes down the fewer chances their are to
mess up mechanics.

Is this a group that needs to be worrying about perfect stat weights? No. Gaining a few percent more HPS/DPS is not going to help them. Despite what Grumpy said, 1% more is not 1% better -- if you're wiping to Doom Wells/Shared Fate on Gorefiend that early in the fight then 1% more DPS, 5% more DPS, and 20% more DPS all have the same effect: none.

And let's look at another statement in that block of quotes: "Casual or not, there is no excuse for not being the best you
can." Yes, yes there is. Time is an obvious one -- I'm sure most people remember the horror of MoP dailies and rep gated valor items from multiple factions. Overload is another one -- pretend you're a newer player trying to get into normal mode and figure this whole raiding this out. You're given one of two "checklists":Checklist A1. Get Augment Runes from the AH for now, work on getting the permanent Augment Rune from Tanaan rep. 2. Watch boss videos/guides for the first four bosses in HFC.3. Figure out what your ideal stat priority so you can evaluate which pieces of your current gear are weakest.4. Buy/craft some 715 items for those weakest slots.5. Gem/enchant your gear with your best stats.6. Figure out what your best stat is and get 125 stat food of that type, since feasts aren't optimal.7. Research your set bonuses so you know which tier pieces you want and whether it's worth using some lower ilvl pieces.8. Practice your skills in Proving Grounds.9. Show up to raid and do your best.Checklist B1. Watch boss videos/guides for the first four bosses in HFC.2. Gem/enchant your gear.3. Practice your skills in Proving Grounds.4. Show up to raid and do your best

In a perfect world, is someone who does all of Checklist A better off? Sure. But ask yourself this: what is the actual *practical* difference between the two in terms of end result? I'll give you a hint: it's very small and insignificant for the vast, vast majority of Normal raiders. And Checklist B is a lot more likely to actually get done for said players while Checklist A is likely to overwhelm/discourage them.

Before we conclude, let's take a look at some actual numbers about the difference that ideal vs non-ideal stats makes. I'm going to look at Shadow Priests for two main reasons:

2. Auspicious Spirits has some crazy secondary stat weights with Crit being worth more than Intellect and Mastery being less than half the value of Crit. In other words, it is arguably the spec (or at least one of the top 2-3 specs) where stat weights matter *most.* 90% of specs or whatever are going to see a much *smaller* difference than what we find here.

So what we're going to do is pick out our ideal Mythic items from BRF and then pick out our least ideal items from BRF and see how much the best set possible versus the worst set possible changes our DPS. They're all 700 ilvl (which is why we're doing BRF) and I'm going to ignore set bonuses for the moment (since in theory people would have four set either way and this way we can skew the results even MORE). I'm also just truncating (rounding down, basically) the decimals for the sake of time -- also note that since we have sixteen slots that this means the maximum possible impact is a whole 16 int when we're dealing with thousands of int.

Yeah. So even with intentionally picking a spec with insanely skewed stat weights, intentionally ignoring tier (which is both a normalizing factor between the sets and an overall boost in power to both which decreases the percent difference), and intentionally/deliberately picking the absolute worst and absolute best sets possible to compare...we got less than an 8% difference.

90%+ of specs, even using this deliberately awful comparison, would probably see more like a 4-5% difference worst case. And a person just randomly picking gear without regards to stats would see more like a 2-3% difference. Maybe less.

Optimizing gear matters far, far less than most people think.

There is one last thing I want to point out/acknowledge. A lot of Normal raiders may not be willing to improve -- they simply don't care enough to fix basic "rotation" problems or talent choices. And for those people perhaps trying to optimize gear is the only way to get *any* improvement of their output, even if it's only 2-3%. But, of course, Normal raids are not tuned to the point where that 2-3% even matters -- sheer ilvl increase over the course of a tier is going to result in a much larger improvement and the vast, vast majority of wipes are solely going to be due to massive screw-ups of mechanics.

This leads to perhaps a paradoxical conclusion: the people who are capable of realizing how small of a difference gear optimization makes are also the only people for who the gear optimization even matters.

I started raiding in MoP and it was (quite rightly) essential to turn up with every item enchanted and gemmed. I was pointed to Ask Mr Robot, and initially followed it to the letter until I realised that changing one item meant it was telling me to re-gem vitually every other item. Then I'd drop another item and be told put the gems back to where they were. It was costing me a fortune. So I only re-gemmed/enchanted if it would give me a 2-3% or better improvement. It made no appreciable difference to my DPS, which I know you won't find surprising, but I did. The penny should have dropped then, shouldn't it?

I have come to rely on AMR too much. There's a lot less to gem & enchant now, and those choices are pretty obvious, but I have been relying on it too much when deciding whether or not to equip a new armour piece. If it has more INT or AGI I'm putting it on now, damn the nay-sayers :-)

Seeing the "working out" about how gear weightings work is so useful: I assume that's all AMR is doing?

My raid level is comfortably at Normal, and I'm getting better at Heroics. I will never be a Mythic raider, my raid awareness simply isn't good enough. Your guide has made me realise where I need to spend my efforts.

"Then I'd drop another item and be told put the gems back to where they were. It was costing me a fortune."

Just as an FYI, you can "lock in" MrRobot so that it ignores changing gems/enchants and only worries about changing reforges. Or could, at least, it's irrelevant now with reforging gone and probably therefore removed.

But yeah, a lot of people were spending a lot of gold to try to get 0.1% more DPS in scenarios where it simply wasn't worth the cost.

"If it has more INT or AGI I'm putting it on now, damn the nay-sayers :-)"

Keep in mind that my point is not that more Int/Agi = always better, but rather than even if you're wrong that the difference is insignificant overall. Technically speaking a socketed item that's 695 is almost always better than a non-socketed item that's 700..but again we're talking less than half a percent here overall.

Trinkets are the main thing to worry about as some of those can be very different, especially in HFC. For example, the hunter trinket from Archimonde makes you do more damage the further away you are...so unless you're good at keeping max distance or whatever then better off using another trinket just for ease of use.

"Seeing the "working out" about how gear weightings work is so useful: I assume that's all AMR is doing?"

Correct. And you can set the exact stat weights you want AMR to use if you want (which the high end people did).

This is true. But I'd bet that an hour on icy-veins and then maybe 20 minutes in front of a target dummy will probably do more for someone's DPS than the hours it would take to optimize their gear to that extent (because they'd have to go pick up a bunch more gear to swap out for "better" secondaries).

Hehe, in reality the situation is MUCH worse. I mean, ever wondered where those "stat weights" come from?

They are determined with software like simulationcraft, which executes a "virtual boss fight" varying the stats and comparing the DPS. And there's a ton of problems with that which are conveniently swept under the rug.....

First problem: randomness, RPPM, crits, it all adds up. Even if you execute the rotation as PERFECTLY as simcraft, there's a fluctuation, which simulationcraft nicely provides you with: the sigma of the DPS distribution. It can be huge, up to a +-8%. You can "get rid of it" by averaging on, say, 100k fights, but when you raid you do the fight ONCE, so you get the full +-8% random fluctuation. This by itself is often sufficient to hide anything else.

Second problem: do you really think you can execute your rotation as well as simcraft while dealing with the fight mechanics? If you're in Method or Paragon maybe I can believe it, for all the rest of the population I don't. And here is the problem: the weigths *depend* on the rotation. Different rotation = different abilities = different weigths.

Third problem: who told you that the simcraft rotation is the optimal one? Well, noone, it's blind faith. Maybe the author messed up, or more simply, he optimized it for a pure patchwerk fight. Unless the boss you're doing is exactly a patchwerk, chances are that the rotation in the simcraft file is NOT the optimal one. And since different rotation = different secondary stats.... well.... you have a problem :)

Overall, just listen to Balkoth: unless you're pushing Mythic progression and you already master your class you can safely go with ilvl > anything and just make sure you don't die in the bad stuff..... dead DPS = 0 DPS, no secondary stat choice can do worse than this. :P

"Even if you execute the rotation as PERFECTLY as simcraft, there's a fluctuation, which simulationcraft nicely provides you with: the sigma of the DPS distribution. It can be huge, up to a +-8%."

I can remember the tears of our hunters in BRF when some got multiple Beastial Wrath resets from tier bonuses and others got none for a long time.

"If you're in Method or Paragon maybe I can believe it, for all the rest of the population I don't."

I would argue that quite a few more guilds than Method and Paragon are at least "close enough"...but that's really a nitpick since even counting those guilds we're talking less than 1% of the overall population, so your general point is absolutely valid.

"Maybe the author messed up, or more simply, he optimized it for a pure patchwerk fight."

Simcraft has actually gotten significantly better at this and actually has multiple fight settings which simulate different environments that you can select...but yeah, none of them are still exactly equal to a particular boss fight (outside of Patchwerk). Especially if you look at something like, say, Gorefiend which was a Patchwerk like phase like 2 minutes into the fight. Which means you don't WANT to be using your three minute cooldowns right off the bat...but you DO want to use 1 or 2 minute cooldowns...and it gets more complicated from there.

"just make sure you don't die in the bad stuff..... dead DPS = 0 DPS, no secondary stat choice can do worse than this. :P"

Please stop trying to make up arguments such as "I am trying to teach these people this". You combined two different posts about two completely different things. One was a post about being fustrated with new players in a raid and the other was about item level not being the best way to tell what someone is capable of. Not even sure how you connected two completely separate topics.

I'll ignore your bashing of the new players, they have to learn somewhere, and move to the topic at hand. Which I am not even sure I even need to as you were so kind In your own post to prove I was right, gear choice matters.

Now to the item level conversation only, which you completely ignored otherwise.

I never said anything about different skill levels in my post. I never said anything about the people in my guild in my post. I was talking about item level and item level only.

Reading comprehension issues maybe? Lets talk item level, not some random normal raid group. How did you even manage to put the two of them together? Really, I would like to know how you manged to put together two things that were not in the same post and had absolutely nothing to do with each other.

I said the best gear for you is the best gear for you and putting on something just because it has a higher item level does not mean it is better.

You can not compare apples to oranges and call it a fair comparison so remove skill from the equation. The better player will always do better when gear is close enough. We are not talking skill, we are talking judging people by their item level.

Now to what the post was about.

Take 1 player, only 1 player, put him in 685 item level well itemized gear with the 670 4 piece tier and then change that 1 players gear to not have the tier set and have pieces with poorly itemized stats instead of good stats and no tier set with a 705 item level.

Guess which gear he is going to do better in? The 685 gear.

Remember, we are talking the same player, which means the same skill level. Just want to stress that for you. We are talking a single individual here.

Skill will always outplay gear, this is true. Me doing 99% of my potential will double up someone in 705 gear doing 50% of his, but that is not what the post was about. I was talking about not judging people based on gear and said nothing about skill levels. It is more inspired by someone saying I could not do 40K at my item level when I did 56K single target the other night. (lucky procs admittedly) 685 should not lock me out from content because people are looking for 40K. Do not judge a book by its cover.

I was going to dig up the numbers to use for my example but you supplied some for me with your priest to prove my point so I won't. You proved it for me, thank you. Choosing the right gear matters. Anyone that says otherwise is misinformed.

PS: That post was not an instructional post for my guild members. None of them read that blog, none of them even know it is there. I do not tell them on purpose as it allows me to be brutally honestly and not feel I need to sugar coat things.

Not sure where you put the two together and thought that post was me trying to help people in my guild. It was not. It was a general knowledge post which, disagree all you want, is completely 100% based on facts.

Gear matters, tier matters, stats matter. And 5-8% better is still 5-8% better and I can't believe you would tell people not to worry about getting better? I thought you were a mythic raider. If you can get 5-8% better by sometimes as simple as wearing the right gear, why wouldn't you?

Take 1 player, only 1 player, put him in 685 item level well itemized gear with the 670 4 piece tier and then change that 1 players gear to not have the tier set and have pieces with poorly itemized stats instead of good stats and no tier set with a 705 item level.

Guess which gear he is going to do better in? The 685 gear.

I don't think you're right about this. You still get "more" stats from the higher ilvl gear, and that is often enough increased value to swamp the better stats, or make the comparison much closer.

Non-optimal stats still have value, and an increased quantity of non-optimal stats can be better than a lower quantity of optimal stats. 3 dimes are worth more than 1 quarter.

You're right that item level is a weak proxy for output. But you're wrong in attributing it to stat weights. The greater contributor to output is skill. However, skill is hard to determine at a glance, while item level is easy, so item level is the mechanism used as a filter.

Rohan is correct. I *know* you're incorrect about this Grumpy, mathematically. I've done the theorycrafting behind it ( http://talarian.blogspot.com/2014/12/wow-why-rando-secondary-stats-matter.html ).

At ilvl 630, a 15 ilvl upgrade is *always* preferable, *regardless* of stats, because as Rohan says, no secondary is entirely useless. Heck, for many classes that ilvl gap is only 10. And the gap gets smaller the higher the ilvl because stats increase exponentially over ilvl budget.

However, I will grant you that set bonuses throw a kink in the works. I know in the past some 4p set bonuses were worth a good 50 ilvls (Cata comes to mind), to the point where Blizzard would nerf them the next tier because they didn't want folks holding on to them. But not all 4p bonuses are made equally. Some classes like theirs more or less than others. Enhancement, for example, last tier's bonus is a 1% - 2% increase in DPS. I'd dump that in a heartbeat for 705 gear.

Trinkets are another iffy zone, I grant you (see Healers and the Candle).

The only other thing that throws a wrench in the works are sockets, and those you can generally consider equivalent to Warforged at ~700 ilvl unless you toss an epic gem, in which cast, treat it about 10 ilvls ( see math here: http://talarian.blogspot.com/2014/12/wow-secondary-wars-socket-strikes-back.html ).

Stat weights only really matter when you're trying to figure out side-grades (+/- 6-10 ilvls per piece of gear). Otherwise? ilvl is almost always king for determining maximum possible output.

"I'll ignore your bashing of the new players, they have to learn somewhere, and move to the topic at hand."

If you think I was bashing them then you missed my entire point. I see a *lot* of people (including, frankly, you) who are trying to get aspiring Normal mode raiders to act like Mythic raiders. But that's not what they need to start and it's only going to overwhelm the vast majority of them -- see the comparison between those two checklists I provided. My message to those people is "Don't worry yet about picking optimal gear, just take higher ilvl stuff and focus on working on playing your character better." Simplify their life. Focus on the fundamentals that offer the greatest return (which, to be clear, do not include Augment Runes).

"I never said anything about different skill levels in my post."

Uh....

"He was a damn decent player to say the least. As I said, it would be easy to just write it off and think he was doing good because his gear was carrying him but that was most definitely not the case. He knew how to use the gear he had.

There could have been others in there with the same item level as him and they could have done poorly because they did not know how to get the most out of the gear. I see that all the time."

Does that not count?

"Guess which gear he is going to do better in? The 685 gear."

For 90%+ of specs, that is wrong. The only exceptions are going to be obscenely powerful tier bonuses which are not the norm.

"It is more inspired by someone saying I could not do 40K at my item level when I did 56K single target the other night. (lucky procs admittedly) 685 should not lock me out from content because people are looking for 40K. Do not judge a book by its cover."

The problem is that most people with that item level are not going to be doing 40k. And the PUGs have no way to verify that you, in fact, are capable of doing so short of actually inviting you and potentially wasting their time.

"If you can get 5-8% better by sometimes as simple as wearing the right gear, why wouldn't you?"

*I* would. And do. But the time it would take most Normal and Heroic raiders to learn about wearing the right gear and all could be spent getting 10%+ better (and often more like 20-30%+ better) by learning/practicing their class.