Asking the Right Questions

It occured to me recently, after watching Brian Burke bloviate, Jay Feaster deflect and Steve Tambellini ramble during their respective year-end press conferences, that the true issue facing mediocre (or worse) teams is not whether an executive has all the answers – it’s if he has the wherewithal to ask the right questions.

The goal of managing a hockey team is simple – gather as many good hockey players together (for X amount of dollars) as possible. Unfortunately, there is an innumerable number of blind alleys to go down in pursuit of that destination. Player performance fluctuates year to year due to chance, circumstance, injury and career arc. The shifting free agent and trade markets continually alter the value of player assets. Throw in the fuzzy stuff both decision makers and fans tend to attach a lot of significance to (leadership, chemistry, character, truculence etc.) and you have a baffling miasma of factors and gambles to sift through every off season. Which is why if a management team isn’t endlessly curious about its outcomes or ruthlessly honest about its assets, it’s possible not only to get lost in the desert but also be driving in entirely the wrong direction.

In this series of posts, I plan to look at how the wrong questions are easy to ask and difficult to avoid.

Larger Samples are More Meaningful Than Small Ones

I’ve noticed fans, coaches and managers of lousy teams tend to get caught up talking about relatively small stretches of games when the team tasted some success. This happened a few years ago with the Edmonton Oilers for instance. They were still terrible, but nevertheless reeled off a month of better than average results at the end of the year. This was marketed to fans by the organization that, clearly, things were looking up. Since then, they have finished 30th, 30th and 29th in the league.

I argued against this type of thinking during Feaster’s first (half) season at the wheel when the Flames were the hottest team in the league for about two months. Similar narrative building is happening again in Calgary this off-season: both Sutter and Feaster have talked about how good the Flames were when half the line-up was made up of AHLers and kids due to injury. The obvious solution going forward, therefore, is to make the roster indiscriminately younger. Younger means more wins, right?

(Imagine if it was that easy)

Intuitively, it seems to make sense to seek out periods of success to try to determine what was different/better about the team at the time and then try emulate those circumstances. Unfortunately, the truth is just about every team experiences periods of better than average results as a matter of chance. Perhaps the schedule was easy, perhaps the PP got hot for a few games, maybe the goalie was standing on his head for awhile. Think about Edmonton leading the NW division for the first six weeks of the year or Minny leading the West at the mid-way point of December, for example.

The challenge is to be consistently skeptical of hot streaks when there is a longer history of mediocrity. More data is almost always more powerful than less. If a club has a few weeks of results that is apparently out of line with an established skill level, then it’s important to examine the information from those games with the goal of discovering how sustainable – how true – those results were. In essence, a bad team on a good run should try to debunk its own (briefly held) success in an effort to determine if the improvement is real or not. If a club has established that it is average or worse, that should be the guiding assumption, even in the face of apparent, sudden improvement.

This is, of course, exceedingly difficult to do for fans and decision makers alike. The human brain looks for positive results and easy, causal explanations for things. Coaches and GMs will naturally focus on the good stuff because everyone prefers to win rather than lose and to be right rather than wrong. And during any transient outburst of success, there is bound to be some conveinient, intuitive explanation for the change in fortune.

Myth Making

Last year for Calgary, it was Darryl Sutter’s departure and the subsequent change to the Flames culture that caused the improvement. This year, it was the energy and exuberance conferred by the rookies. In Edmonton, it was the emergence of the kids and the rehabilitation of Kahbibulin. In Minnesota, new coach Mike Yeo had the team playing together like never before! Im not sure what easy explanations popped up in Toronto when they rushed hot out of the gate this year, but I’m guessing they have disappeared into the ether by now.

Even bad teams have good stretches of games. And those outcomes are not necessarily an indication that something has fundamentally changed, that the club has truly improved. The tendency to weight those periods as more meaningful than others and to fabricate fuzzy but plausible sounding explanations for them should be avoided.

The question for average-to-below -verage teams should never be "why are we occasionally good?" Rather – "why are we so often bad?"

So I agree with you Kent. We learn a great deal more from our failures than we do from our successes.

I am going to say though that a GM being brutally analytical in the media is a quick ticket to dismissal. Whether we here like it or not, the majority of casual fans want to hear things are getting better and need that ray of hope. The GM, in his public conferences, must give that.

What I like about what Jay has done here is that he’s spread the information gathering out and hired more people to give their view. In the end the decision will ultimately be Jay’s but he has many more people giving input that Darryl ever did. Why this is important is that I choose to believe that these people are not just sycophants. Most of these people have in depth knowledge of the sport and help steer the ship in the right direction. I doubt very seriously that the people around Jay are saying “Buy up all the young players you can and we’ll get better”. However Jay does have boundaries. The owners, understandably, don’t want to lose more money than they have to. This probably puts Jay in a sticky situation regarding #12.

What am I rambling about? Looking at Jay’s performance I’d have to give him a B so far if I were to grade him. Yes he has talked about small sample sizes and used that to point to how the Flames will be successful. However I think that is a public line. His passing grade from me comes from his decisions to move some players out. Bring in some players, who are younger, that are talented. Slowly start to change over the old guard. With a couple missteps along they way (I’m looking squarely at Babchuk and Modin).

As for the other GM’s, I cannot comment on Burke. Tambellini doesn’t seem to really “get it”. That’s about all I know there.

Wasn’t a year end collapse in 08-09 the problem? If I recall correctly, Edmonton was in a solid playoff position in march and a bad stretch dropped them out. It’s irrelevant I know. It just seems to be forgotten.

I’m sure with what’s going on in Edmonton and Calgary isn’t everyones cup of tea. Have to think a couple players gave their team permission to move, or just flat out asked to be traded in the coming off season.

Nice piece Kent. It is truly amazing that some executives resist the wave of change happening with the understanding of hockey. Who wouldn’t want their job to be easier, have more success, look smart, and not even have to spend much to do it?

I suppose it’s a consequence of the tiny gene pool that is the old boys club running teams and the league.

This post and other recent ones (especially j.willis piece right after oil won the lottery again make me wonder if and if so how often oil management, players and coaches read oilersnation, listen to the gregor show, and maybe even roam sites like hfboards for entertainment..
I have tought in the past if I were to win a major lottery like a recent 600 million dollar win in the US, I would invest major coins into Katzs Oil Tycoon if he agreed to make Oilersnation leaders in the management team, give STU the gm position and I would be that guy in the office who plays the role of nobody exactly knows what he does.

I think a related issue is the propensity for results based thinking among the Tambellini and Burkes of the world.

I keep flashing back to Burke’s quote on the Sloan analytics panel about Ryan Kesler. I’m paraphrasing here, but he basically said that he was watching a game featuring Kesler’s NCAA team in which Kesler happened to have a particularly strong shift, and Burke decided right then and there that he was going to take Kesler in the 2003 draft.

Now, he turned out to be right about Kesler, but the process of how he reached his decision stunk. Drafting a guy based on a single shift is absurd.

Of course, that’s not the narrative that Burke believes. Instead he brags about how he just knew about Kesler and uses that as evidence for his superior hockey intellect.

…”It is truly amazing that some executives resist the wave of change happening with the understanding of hockey. Who wouldn’t want their job to be easier, have more success, look smart, and not even have to spend much to do it?

I suppose it’s a consequence of the tiny gene pool that is the old boys club running teams and the league.”

Correct on the tiny gene pool. And the lack of qualifications to manage somewhat difficult business arrangements. Change is always hard to accept by people that are operating beyond their capabilities. Another factor is player agents are brilliant compared to most GMs. Additionally, the player agents pull the strings at the NHLPA. Look what happened to the salary cap that was supposed to bring financial certainty to the NHL. Players are being paid more and many hold no move/no trade clauses in their agreements. Brilliant!