Mike Smith: Analysis at its Simplest

Moneypuck -Contributor IJuly 31, 2009

The business of sports rarely stand still. With hundreds of millions of dollars available to be had, the need to be better is always at hand. The "edge" is always looked for from pro sports executives. Any little tip, method or thing that could propel them over their competetion is valued highly in the industry.

So enter hockey, and more specifically hockey analysis. There is a glaring hole in hockey statistical analysis. Behindthenet's Gabriel Desjardins goes as far to say ," "Hockey is so far behind the other sports on the analysis side that it has a bit of a wild west feel to it, and a sense that some random blogger somewhere might have figured out something that many GMs and coaches don't know."

Throughout the next while, you will see me try and detail the hockey statistical world as much as provide my own analysis, back on-topic though.

With such a somewhat easy entry point, to what one would think would be a very enticing speciality, that if done properly, could easily give NHL teams that much-needed "edge" in terms of evaluating players, its mind-boggling though to who the leading statistical advisor for the NHL is.

Meet Mike Smith, the Chicago Blackhawks GM from 2001 until his firing in 2003. Mike is a founder of the company called Coleman's Analytics. An analytics company who have several NHL teams as clients.

Smith claims that " This season, 2008-09, we have five clients. All five made the playoffs. Last season, we had six clients; five made the playoffs. (That is 10 of 11, if you’re counting). The first two seasons, 2005-06 and ’06-07, we had a total of nine clients and seven made the playoffs. (That is 17 of 20, if you’re counting).

One would think that with such a huge "success %", Mike may really be onto something and there may be a correlation.

Lets hear what kind of analysis this man and Colemans produces:

What makes up these different games? One is when a game is on the line. Another is when the game is out of reach, i.e., a team has a three goal lead or greater. Two others are shorthanded and power play situations. Yes, you can probably think of others: How about power play situations when the game is on the line, or when it is out of reach?

We are able to show how each player performs during each of these “new games.” How does the player rank league-wide in scoring when the game is on the line? How is his plus-minus? What are his linemates’ numbers like? Does his performance increase or decrease as game situations become more critical? And so on.

There is one player, who shall remain nameless, who has never been above the bottom third in scoring when the game is on the line and his scoring rate decreases as the game situations become tougher - and he has never made less than $5 million. No, it’s not Alex Ovechkin, nor Teemu Selanne nor Sidney Crosby.

And here's another gem:

Another example: How do you rate the best goalie? Is it save percentage? How about save percentage on close shots? Or maybe save percentage in the third period when the game is on the line or save percentage in shorthanded situations on close shots when the game is on the line…You get the point. More likely, it is a combination of several analytics.

Nothing will make me smash my head more than these two words:

1. Plus-Minus
2. Clutch

Because honestly, what is Mike's sample size in this whole matter? What is it a couple hundred of minutes to determine which players are truly "clutch" or not?

Its been proven " time and " time again in baseball that while clutch exists, over a large sample size the real significane seems to die down and really make a partial impact.

Its the small sample sizes (such as Johan Franzen's recent postseason success, in a mere 39 games people are claiming him as a guy who can "raise it to the next level in big moments" already).

Now clutch is a hard one to sell, its a hard one to sell to long-time sports watchers that clutch really isn't that significant, but to go as far as to use Plus-Minus to make detailed analysis of players is beyond me.

Whats even scarier is GM's are buying and using this information. Does nobody understand the fact that plus-minus isolates absolutely nothing about a player? Anybody?

To use that in connection with a small-sample clutch factor could only mean Mike Smith and Co. are selling absolute crap to NHL Gm's.

However Smith doesn't stop there, he goes on to seem outlandishgly cocky about his work:
Hey, maybe there is no need for the GM to know that the player he just traded for, who makes more than $5 million, scores in the 27th percentile when the game is on the line. Really, why would a GM want to know this? His owner might. But then, maybe it is valuable for the GM to know that the defenseman he can get for a fourth round pick at the trade deadline, who makes less than $1.25 million and has another year on his contract, is in the 81st percentile for plus-minus in the third period when the game is tied.

I'll tell you what Mike Smith, you keep your Maxime Talbots of the world, I'll gladly take Marian Hossa and smile every second I do it.

I mean seriously, if you really, really want to use plus-minus, then I highly suggest On/Off Ice +- (whcich shows the goal differential of the team for every 60 minutes the player is on the ice compared to 60 minutes off the ice) or Corsi (which is essentially like +- except using shots, including missed and blocked, it widens the sample size and takes the goalie out of the equation).

While I don't recommend ever using any sort of +- as a conclusive stat, my god if you're going to use it do it smartly.