Inspired another blog and chat on #metabox I thought I'd just leave this here.

The phenomenon was first experimentally observed in a series of experiments by David Dunning and Justin Kruger of the department of psychology at Cornell University in 1999. The study was inspired by the case of McArthur Wheeler, a man who robbed two banks after covering his face with lemon juice in the mistaken belief that, because lemon juice is usable as invisible ink, it would prevent his face from being recorded on surveillance cameras. The authors noted that earlier studies suggested that ignorance of standards of performance lies behind a great deal of incorrect self-assessment of competence.
This pattern of over-estimating competence was seen in studies of skills as diverse as reading comprehension, practicing medicine, operating a motor vehicle, and playing games such as chess or tennis. Dunning and Kruger proposed that, for a given skill, incompetent people will:

fail to recognize their own lack of skill

fail to recognize the extent of their inadequacy

fail to accurately gauge skill in others

recognize and acknowledge their own lack of skill only after they are exposed to training for that skill

Dunning has since drawn an analogy – "the anosognosia of everyday life" – with a condition in which a person who experiences a physical disability because of brain injury seems unaware of, or denies the existence of, the disability, even for dramatic impairments such as blindness or paralysis: "If you're incompetent, you can’t know you’re incompetent.… [T]he skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is."

TLDR: It's not the dice, it's you, and you are too ignorant to realize it.

alright, grab a pack of zons and throw them at some dwarves. or, even better, flings.

Posted by mrt1212 on 2016-08-06 00:40:20

What does that have to do with recognizing your own skill and how you analyze it and what you attribute your record to?

Come back later when you've sobered up and can process what's happening.

Posted by B_SIDE on 2016-08-06 02:53:49

The effect is pretty well documented. The lower your ability, the greater the margin by which you overrate yourself. Because most people who are truly terrible at something actually consider themselves "average". Average folks rate themselves a little higher than average, typically. But people with very high skill also make an error here, usually rating themselves higher than average but not as high as they should.

Dunning-Kruger is one reason that people across the entire spectrum of skill might benefit from coaching.

Posted by Wreckage on 2016-08-06 03:42:21

I did read up on this quite a while ago because it came up on the site before. Kind of the first thing wiki flaunts in your face is that this isn't being seriously discussed as a thing in academic circles, which should give you pause.

I can largely agree with the concept on an empirical basis that it is possible to misjudge one owns abilities and that this tends to be related to skill in the same field.
These assumptions are way too absolute tho and the way they were experimenting this was build on way too many false assessments of what skill really is.

Unsurprisingly the results couldn't be replicated in a statistically meaningful way.

The original bias was basically that the tester would be able to objectively determine somebodies skill.
The test subjects would improve their abilities to evaluate their own abilities correctly after they had been evaluated themselves.
But this is actually wrong, they were not able to better evaluate their skills but able to better evaluate how they would perform in the way the testers evaluate their skills.

In Bloodbowl that is not really transferable because we don't have to satisfy somebodies arbitrary criteria. We see our statistics and can self-evaluate based on them.

Although a statement like 'incompetent people will fail to recognize their own lack of skill, fail to recognize the extent of their inadequacy, fail to accurately gauge skill in others' is certainly more likely to be true for unskilled coaches, that's basically all it has in common with Dunnings and Krugers ideas.
The thesis itself already fails in that skill-assessment is a skill in its own right, that can be better or worse than the own actual skill.

In BB it's easy to self-assess well but certain psychological conditions need to be overcome and put into perspective.
a) BB is a dice based game. The impact of dice is huge. Therefore the winners of all games are decided randomly.
b) BB is a dice based game. The impact of dice is huge. Individual failures that can be explained away with individually poor dice are not relevant for self evaluation.

There are more criteria but let me elaborate on b) since this could be meaningful to some.
The assessment of b) is, of course, correct. But surprisingly still poor for self-evaluation.

That is of course because our explanations of events end with our level of perception of what is happening in a game.

Just to make an example that should be easy enough to understand for everyone:

Say a coach hasn't spend any thought on turn time management ever.
He continually scores in turn 4 of his offense and develops the for this strategy expected win percentage of 33%.
Although he does seem to do everything right and constantly self-reevalutes his solid offense that he managed to play well and his defense where he doesn't do any worse than his opponents do, he just can't wrap his head around why he keeps losing so often. The fact that he does win the odd game gives him confidence tho. Sometimes he breaks the other guys offense, sometimes the other guy has to score early.
As a consequence he concludes that he often just hasn't been lucky in the way turns had been left in a drive and if he continues to play on like he has done, surely he will be luckier with the turn times. Perhaps he even proclaims some sort of godly intervention to explain things, because there really is no other explanation (from his point of view).

This is interesting because in this example the coach would be able to even recognize that turn time is a factor but simply can't tie it to a level of competence in the game. Time of scoring may seem arbitrary and the act of conscious scoring at a given time even as a sacrilege.

On the other hand there are certainly many examples possible where a coach is genuinely unlucky, especially over a small sample size of games or actions.
Even a good coach may run into the same traps because on any level you will be limited to the scope of your own imagination in describing the game.
Of course he is less likely to do well because his overall competence is higher but he won't be able to improve if he never looks for other causes of his shortcomings than his own luck.

Just to give you another example that is a bit more dice oriented from very early out of my carrier that I often like to quote.

The first team I started on Fumbbl was a Woodelf team. It did in the beginning reasonably well but would soon succumb to injuries and never really recover (lrb4 times). I kept going on using it and found that I would do reasonably well as long as nobody got injured.
But I always had bad luck with the injuries on my key players. I thought it's 2d6 for the AV roll and 2d6 for the injury roll. What influence do I really have about who gets injured when?
I never thought about holding back because obviously if I didn't use all my most valuable pieces to attack a cage I'd lose because of htat.

Although I improved on many different levelss it took me about 2 years to finally come to a point where I began to understand that leaping into a cage to strip a ball and have my wardancer surrounded by six opponents and the subsequent fouls he had to endure every following turn weren't completely random in cause and effect. Although of course each individual armor or injury rolls outcome was.

Posted by thoralf on 2016-08-06 05:24:30

I'm not sure what statistically significant replication means in this context, but DK's results are quite robust:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2702783/

If that is not enough, follow the American presidential race.

Posted by paradocks on 2016-08-06 05:43:15

I have the DK effect written on my kitchen wall, it helps prepare me for the day

Posted by pythrr on 2016-08-06 11:35:41

TL;DR people are morons

Posted by Wreckage on 2016-08-06 13:26:00

Well, this for instance is an opposing study,
https://deepblue.lib.umich.edu/handle/2027.42/39168

But I can't emphasize enough that being tested by subjective criteria isn't the same as being skilled or being not skilled.

Since a simple test can never fully evaluate somebodies skills it's only natural that if people assess their skill correctly they would assess themselves better than their score under actual test conditions are.
On top of it there is social and work ethical pressure to self represent in the best possible way. So when it happens it's as likely to be a social condition and not a cognitive one.

Posted by thoralf on 2016-08-06 14:36:38

Burson 2008 is discussed in the paper I linked, Wreckage. It does not even dispute the DK effect. It only questions the usual law-like interpretation, which abstracts the difficulty of the task to perform and guesstimate.

Posted by Lorebass on 2016-08-06 22:30:01

You realize you just used the word 'guestimate' in regards to a scientific paper.

Posted by thoralf on 2016-08-07 02:04:03

DK refers to an effect that can be characterized as a guesstimate:

"[A]n estimate made without using adequate or complete information, or, more strongly, as an estimate arrived at by guesswork or conjecture."

https://en.wikipedia.org/wiki/Guesstimate

There are more than 10k hits for "guesstimate" in G Scholar.

Posted by Nightbird on 2016-08-07 05:45:10

ZzZz...

Posted by fidius on 2016-08-08 02:57:56

This research is commonly referred to in the skeptic blogs, which naturally are dedicated to putting down theists. In America this extends naturally into bashing the Right, so the blogs tend to be thinly disguised havens for Leftist politics. The irony of course is that the findings can just as easily be used against them wrt opinions on overpopulation, climate change, minimum wage, and all manner of political arguments that likewise defy rational analysis. Many of these articles unwittingly descend into self-parody pretty quickly.

Not saying that's what you're doing here. DK is unquestionably applicable to BB, and any coach who takes it to heart will instantly become a better coach. I've already taken it to heart, unfortunately, and yet I'm still average, so my upside is obviously limited.