The Playoffs are a Crap Shoot

Moneypuck -Contributor IAugust 20, 2009

They are the epitone of sports, the peak of greatness, the place where people define their legends. It's a time where everyone is watching, where everything is exxentuated, every play vital and every mistake critical. In hockey names like Mark Messier and Claude Lemiieux are some of the most notable of athletes who have made their mark in hockey history through the playoffs.

We hear the notion all the time that, "It doesn't matter what you do in the regular season, get it done in the playoffs!" and to some degree that is true, because in the current system a team is awarded for winning four 7 game series and thereby are named a champion. However here is the issue, is this system that the NHL uses to determine a champion, actually a valid measurement? We see it used all the time in sports, basketball uses the exact same format, baseball uses the five game series and follows it up with 7 game series in the NLCS and World Series, whih Oakland Athletis General Manager Billy Beane says about the format,

""My 'expletive' doesn't work on in the play-offs. My job is getting us to the play-offs. What happens after that is 'expletive' luck."

That quote is straight from Michael Lewis' Moneyball, Beane has never been one for politial correctness but he makes his point at least. The NFL uses a one and done format as well, for obvious reasons (fatigue and TV) but it all comes back to the big point, and is that at the end of the day, did the best team win the championship? In my opinion the answer is no, and here is why.

First we have to debate the topic of does the best team win every game? If you've watched enough sports and specifically hockey, I would hope you say no. It's common logic in hockey that just because Team A beat Team B in one game doesn't make Team A better than Team B. If Team A is the best team in the league, they will win more games than the other teams usually, but they won't beat every team all the time.

The San Jose Sharks led the league in 08-09 with 53 wins and a .713 PTS% which means they got 71.3% of their possible 164 points. In 07-08 it was Detroit with 54 wins and .701 PTS%, in 06-07 it was Buffalo with 53 wins and a .689 PTS%. While these are all great acomplishments, none of them are perfect are for that matter even close to perfect.

For those who want to argue regular season isn't as important as the playoffs, this year Pittsburgh went 16-8, in 07-08 Detroit went 16-6, in 06-07 Anaheim went 16-5. Again these are all great records, but not perfect so if these teams who won the playoffs were indeed the "best team" (which I will debate) they still didn't win every game in the playoffs or even 90% of them. Then can we all reasonably conclude that the best team doesn't always win every game? If you can answer yes, continue reading.

Now comes the hard part, since the best team does not always win every game, how do we know who the best teams are? One would think the easiest way would be to look at winning percentage or points percentage, however the sample of games come into question. Teams go on rolls and slumps all the time, there will be many stretched when teams win five out of six or lose four of five, so is five or six games a fair evaluation of a teams abilities? The Montreal Canadiens in 08-09 started off the month of October going 7-1-1. The Bruins ended the month 5-3-3, if the season ended in October the standings would've looked drastically different than in April, so is a one month sample a good evaluation of a team?

Obviously the answer is no, so is the 82 game schedule even a proper evaluation of a teams abilities? The answer is probably no. In the 08-09 regular season, the average amount of shots a team would deliver or recieve is 30.21 per game, which averages out to 2477.22 over a full season. Using a theory called binomial variance, which is a statistical theory that says in a yes or no stat, such as a shot on net that has a yes or no answer, being goal or no goal, that there is random luck based on the sample size. If a shooter shoots 30% on ten shots, thats not enough shots to clarify he will be a career 30% shooter, so we do square root of (X (1-X))/N with X being the shooting percentage and N the total number of shots.

For example on three goals out of ten, we can conclude with 95% certainty, which is using a theory called standard deviation, that the shooter could have shot anywhere from 5% to 60% on those ten shots due to just random luck. Thats the issue of small sample size, increase that to 1000 shots and the results are 27.10% to 32.90%, obviously a lot more certain of a result but yet still not perfect.

To the point however and to try to make things as simple as possible, on the average 2477.22 shots per season for or against per team in the 08-09 regular season, and with that same average team scoring 2.85 goals per game which translates to 233.7 goals for or against, an average teams for or against shooting percentage would be 9.43%. Now using that handy dandy binomial variation we used before, under 2477.22 shots the variane dictates that we can with 95% confidence that due to random luck, the teams shooting percentage on those 2477.22 shots could be anything from 8.26% to 10.60%. On that volume of shots, that turns to a goals range from the original 233.7 for or against for the average team to anywhere from 204.62 to 262.59. Again thats for goals for and against.

Using something called Pythagorean Expectation, we can conclude that the range in that uncertainty can come out to a maximum of five wins and ten points both for or against, so a maximum of ten wins and twenty points due to random luck is essentially possible!! Now by possible, I mean in the realm of possibility and is extremely slim, so its nowhere near probable. The point though is to illustrate how drastically luck an effect a team. Even though probably the majority are within one standard deviation, that being 68% of the population and are probably affected by three to six games due to luck, which if you think of the amount of one goal games in the NHL, (actually approximately 72% of NHL games this last year were played at a 1 goal margin or tied) if you think of the lucky bounces, fluke goals or anything that could change a score and the balance of a hockey game, three to six games being dictated by luck in a 82 game schedule is entirely possible.

Making a rough estimate, using the 08-09 numbers, to eliminate luck completely we would need teams to play just as a estimated guess, around 800 games. Something that is obviously nowhere near logically or physically possible. However this is only due to uncertainty in shooting percentage, theres passing perentage, faceoff percentage, special teams, these all have uncertainties, all have fators of luck as well as many other fators of the games like turnoves, bad calls etc. that are influenced by luck. albeit all of thes things aren't as important as shooting/save percentage, mainly due to the latter's straight correlation to goals.

So if you need such a huge amount of games to nullify the luck factor completely, an we in any reasonable sense, conclude that a seven game series is a good measurement of a team's abilties? Heck no! I don't know who came up with the seven game series idea, because it has seemed to catch on everywhere in pro sports and is common practice as a way to truly see which of two teams is better, but it's completely wrong.

We've seen these studies in baseball time and time again that the best team will barely win 60% of their playoff games if even. The critics will soon say, "The best team is one who can win under pressure." In which I counter, if they can win under pressure, why don't they win every game or the vast majority of games under pressure, like I eluded to at the beginning of this piece. If a team can win under pressure and luck is not a factor, why not just make every series one game, that would still be a fair evaluation of a team right? Right?

You know deep down the answer is no, and what people may have a harder time grasping, is that seven game series are not a good tool at all for showing which team is better. You would hope that when the games really matter that when teams are trying their hardest it would eliminate luck, but it doesn't it's still hockey and there are huge elements of luck in hockey no matter the effort level of the players or the pressure of the situation. A large part of this is due to the low scoring context of the game, where one goal can mean so much espeially in a decreased scoring context in the playoffs, that if the other team gets one lucky break or so it can swing a seven game series dramatically.

Note: Error in the study. While shooting percentage is a yes or no stat, and it an be used as a binomial it is not a perfect binomial. Let me use an example, the most common reference to a binomial if the flipping of a coin, which has a yes or no answer in heads or tails. Pretty simple right? Well heres the kicker, what if on the first fifty flips you do the flip from the same starting spot such as heads up, with the exact same amount of force and catch it the same way, but on the next fifty flips you change the starting spots, use different amounts of force per flip and catch it differently each time. Wouldn't that change the results even slightly? Thats the issue with shooting percentage, the results are goal or no goal but the shots come from everywhere and differently. Shots are different but their results are the same, the analysis isn't broken, but this "shot quality" could as always, create some staistical noise that could lean the analysis a little extra in any direction. Back to the post though.

So just how much luck is in a seven game series? Well if you've been convinced yet that the best team doesn't win every game, thats a good start, now here comes another tricky theory, in that NHL scoring is a random distribution, in other terms it is Possion , NHL scoring is rare, random and memoryless. NHL scoring happens very few times as opposed to other events, such as passing, hitting, and turnovers. It is random, it is not like baseball which is a Markov chain with events that pile up on top of each other that an increse or lower a team's chances of scoring a run (Ex: With men on 1st and 2nd with a 2-1 count and 1 out, a teams chance of scoring a run is X but with 2 out its Y). There is no such situation in hockey, you can be cyling the puk and dominating the opposing team in their zone, and then one quick turnover and you may be the one scored on. Memoryless means that when a event like a goal, happens it does not influence the next series of events.

Thats the one that people would love to argue, that momentum would indeed change that and make the team who scored the goal more likely to score again or prevent the other team from scoring. That is an unquantifiable argument though so its hard to argue either way, in hockey though after a goal the play resets and theres a faceoff at center, so unless a team went ahead by one in the final minutes of a game, theres no obvious evidence that either teams chances of scoring changed because a goal was just scored. However in baseball, in some instances this is true. When a man is on third and the batter drives him in with a double, he just helped his teams chances of scoring another run by standing at second base. Hockey is random, if you check Ryder's Poisson paper on page four you will see an almost constant pace of how much goals are scored per minute per period, with the small fluctuations likely due to luck and uncertainty in shooting/save percentage.

Sure some goals may come in blowouts, where teams effort level may or may not be the same as in a close game. However, how do we know that some players aren't trying in blowouts? Just because this may apply to some, it may not apply to others and because we can't quantify effort all we know is what the facts say, and that is hockey scoring is mostly random and unpredictable.

So with that in mind we get back to the topic at hand, the playoffs. If scoring is unpredictable, and there is so much luck in hockey, how the heck do we know how to evaluate teams for the playoffs? Well lets talks basics for a second, how do you evaluate a team after hockey game has ended? You look at goals, specifically goals for and against, if that is a positive then the team did well. That essentially is how one evaluates the talent level of a team, by the direct measurements of how it wins games. Since as stated above, scoring is random, then the goals for or against for a team should state the talent level of a team. In order to do this, you use a basic Pythargorean formula:

Goals For ^2/ (Goals For ^2+ Goals Against ^2)= Predicted Winning %. This method had a 93.5% correlation to actual results, more adavnced methods such as the aforementioned Poisson can get up 94.3%, with the likely missing results could be anything from effort level, to goals not coming at the right time or luck. The point being that GF and GA of a team have a absolutely huge predictional value for wins and losses.

So for example, if a team like lets use an example, the Montreal Canadiens, who scord 249 goals and let up 247, you would do 249^2/ (249^2+247^2) which equals .504, over 82 games thats 41.33 wins, Montreal's atual win total was 41. Of course this method isn't exact, or reasons mentioned above as there is a small margin of error, but its pretty damn close.

Now for a playoff series, you model a series binomial events. Basically taking a binomial stat, which is a teams expected win% which has a yes or no answer being win or lose, and pitting them against each other multiple times. The formula for that in a seven game series is,

Summation of: PW^4 x (1 - PW)^(z-4) x (z-1)! / (3! x (z-4)!) Where PW is the predictd win percentage, z is the number of games you want the series to end in. However if you just enter the one team's PW you won't get what you need in a 1v1 series. In this situation you enter Bill James Log5 formula of:

W%(A v. B) = W%(A)*(1 - W%(B))/(W%(A)*(1 - W%(B)) + (1 - W%(A))*W%(B))

Doing this you get the PW of when Team A vs Team B. Now plugging it into the playoff series formula, lets say you have a PW % (A v. B) of after pitting A(58.2) vs. B(53.12) you get a expected win percentage for team A of around 62.5%. You do this but constantly pitting the 58.2 and 53.12 up against each other multiple times with a maximum match-up of seven and four to end the match-up. FYI those numbers were from the Montreal Philly matchup of last year, the results means that even though Montreal was the better team going into the series, they will still lose the series nearly fourty out of one hundred times!! Again this is under the assumptions that:

1. The best team doesn't win every game.
2. Hockey is Poisson, scoring is rare, random and memoryless.
3. There is luck in hockey that can influence outcomes.

The first one is again the most critical because if one wants to use the argument that the best teams can win under pressure, then why don't they win every game under pressure, and of course the ultimate point, is a seven game series a good evaluator of being able to win under pressure? Lets try using the same Montreal vs Philadelphia series but make it a nine gamer, the results are Montreal having around a 64% chance of winning, a slight, slight increase. Take in mind that these may not be "true" results because the original PW's of the respective teams may have been a few points off due to the sample size of the regular season.

The bottomline is this, as the title of this post says, the playoffs are a crap shoot, it is a mini tournament full of luck, like every tournament in all of sports. Theres a reason why teams play long seasons, and its to find out whose truly the better team is at the end of the day, and even then you still aren't one hundred percent sure. To take such a small number of games in a series, and use that to define your champion is absurd. It may be how the system dictates a champion, but it in no way signifies who the real best team in the league is.

I can see why the argument that you want to award a champion by seeing who can play with all the marbles down, that you don't want to play a hundred and fifty regular games and whoever has the most wins takes it all. Effort level does go up in the playoffs, its visibly obvious. However, even with all the marbles down, the format is stil not a fair evaluator of the teams's abilities, because even with two teams going at each others throats, its still hockey.

So when you're sitting down watching TSN experts spew their predictions, or you hear your friends try and guess whose going to win the Cup Finals, you may as well not listen, because even if they're right they have no clue what they're talking about if they think they know who will win. Really when it comes to playoff predictions, just take the favorite and pray, and even then you will still lose a fair amount of the series, and its not because the underdog was a great pressure team or the favorite choked like the mainstream belief always seems to be, its luck.

This is not saying that teams who can play under pressure do not exist, and its fairly possible that a seventh seed can be a better pressure team than a second seed, but you can't conclude that in seven games or less.

FYI I am not a San Jose Sharks fan.