The Cult of Deficiency: A Look at ESPN's John Hollinger and His Flawed Rankings

*I wrote this article back in November of 2010 on a personal blog that I discontinued when I started writing for B/R. I thought it appropriate to repost it as it somewhat relates to an article Kelly Scaletta wrote. I took out sections related to November and tried to update where necessary.

“If we fail to anticipate the unforeseen or expect the unexpected in a universe of infinite possibilities, we may find ourselves at the mercy of anyone or anything that cannot be programmed, categorized or easily referenced.”

–Fox Mulder

I’m never quite sure how a trend starts or how an idea catches fire (I, personally, have never launched anything that has received worldwide acclaim and/or a rabid following worthy of both derision and praise), but there has to be something to it.

An epic charisma that sways all parties, an impressive sense of order or presentation that blows people out of their seats, or that lightning in a bottle moment when something just works.

Obviously everyone is entitled to their opinion, but there is a burgeoning force in the NBA world that seems to embody all the above qualities but, at its heart, is the antithesis of what makes the NBA great.

That force is John Hollinger and his minions are the numbers that make up statistics. Using mathematics as his weapon an entire subsection of class of power is formed, lovingly presented in his Power Rankings, the epitome of what is likely his life work: an equation that measures a team’s strengths by the games they play and the stats they produce.

Hollinger, also the originator of another bogus statistical analysis called PER (Player Efficiency Rating), tries to make the game of basketball a predictable and calculable event, which, while maybe having some merits in sports books or a friendly bet (or perhaps fantasy basketball), kind of ruins the FUN and ART of the game. Plus, only ONE variable can destroy all that is built. . .but more on that later.

Basketball is far from predictable. While we see giants of talent on the floor, using their nearly superhero like bodies and amazing skills that far surpass the common man and woman, in the end they are utterly human, almost embarrassingly so.

Well, maybe not Michael Jordan or Kobe Bryant. . .but the rest are run by the same passions, distractions, and ideologies of the fan in the stands: They just have more experience dealing with it on a basketball court and can dribble better. Basketball, while amazingly physical, is also shockingly mental. And mathematics, as deeply as you look into the numbers, can’t predict the human mind.

Mathematics, of course, lends itself heavily to THEORY and ANALYSIS. Entire sections of mathematics are dedicated to this purpose. And numbers can, in many cases, give you an average or a likelihood of what will happen.

The PPG (points per game) average is a perfect use of statistics in basketball. It’s simple and explains a lot: a player who averages 23.3 points per game will probably get between 20-25 points a night with a dip or massive spike here or there, balancing everything out to that mean number.

The number, while sometimes impressive and responsible for hype and a big contract, is simply non-partisan. It doesn’t TRY to explain that player’s effectiveness to a game or a season.

Hollinger decides to try to explain a player’s worth first with the PER (a massive combination of stats equalled out after some long division and subtraction/addition (not a math major here. . .I think in simple terms) and then by his Power Rankings (all the PERs together amongst other things).

Trying to determine the worth of a player is both over-valuing AND devaluing a player at the same time. The non-partisan PPG or its popular counterparts RPG (rebounds), APG (assists), and FG% (shooting percentage) simply tell the tale of what a player will give you, on average (duh), a night. No further examination needed.

For some, 20 points is as easy as walking, while for others, 20 points is a dream only accomplished two or three times a career. In this case, the record of a team plays either hero or villain to the averages. A dude can average 25.5 points per game and 6.3 assists but if his team is 24-41 at some point, it’s obvious his NUMBERS don’t show his value.

Hollinger’s PER out thinks itself. It tells you something, in raw, boring, elaborate numbers that you can see with your own eyes. Example: I once saw TMac during the Magic’s disastrous 21-61 season in which they started the season 1-19, losing 19 straight.

In the middle of that stretch, McGrady, an immensely lazy but extremely talented player with many a scoring titles under his belt, decided to zone out during a game against the Suns in Phoenix.

He never got in the huddles, never spoke to his teammates and when he got the ball, he’d walk around and then slash to the basket and score. It was the most effortless and careless 20 points I’ve ever seen scored. It was obviously, by the numbers, a ‘great’ performance.

But it was clear McGrady didn’t care and, so early in the season, had quit on his team. See, the PER can tell you how much a player is worth on the bench or off but factors such as intensity, will and heart are incalculable.

So, while over-valuing the players’ role and oversimplifying the game of basketball by throwing data at you full force, Hollinger also, ironically, destroys a player’s value by creating the PER.

See, a lot of basketball players, such as a Jared Dudley, JJ Redick, Bruce Bowen, Bo Outlaw or Chuck Hayes, provide little statistical data worthy of analysis. A few points here, some steals, a great pass.

Their impact, the Intangibles (a spin-off from The Expendables) can’t possibly be explained or calculated. While Hollinger tries, his PER is fallible upon creation because, in the end, it only takes one unpredictable moment to destroy the PER.

And that one moment, be it ONE steal, ONE point, ONE block. . .what have you. . .can’t possibly sway statistical data one way or the other because in the world of math and basketball, at least in a hard value sense without emotion or result, ‘one’ is so unimportant and inconsequential.

Example: Game 5 of the 2010 Western Conference Finals between the Suns and Lakers. The Suns had basically sealed up the Game 5 with a key shot with little time left and looked like they had the momentum.

Kobe Bryant, though, had enough time to get a shot up. Kobe, under Hollinger’s PER system, was a lock or very likely to get something done. But what did he do. . .he lobs the ball up and misses terribly. Disaster for the Lakers. . .yes? No.

Ron Artest, someone of low-middle PER, got the key rebound (only his fifth), and put up a layup at the buzzer to not only score his third and fourth points but also to win the game. PER may indicate that Artest is the type of guy you DON’T want to try to attempt a game winner.

But it took the failure of a high PER scorer to propel a weak PER scorer to success. While only one example and in a moment of higher suspense and drama, PER can’t predict the IMPACT of one or two small statistical entries.

Looking at the list of high PER scorers in history, I can only manage one response: NO KIDDING! Wilt Chamberlain. Michael Jordan. Kobe Bryant. Dywane Wade. Tim Duncan. All Hall-of-Fame players who put up consistent numbers and who, upon simple analysis with the EYES, affect games and prove clutch.

No math is required to let me know these guys were or are great. Oversimplification. In fact, by looking at the top 30 all time PER leaders, it’s the players with high PERs that are the OPPOSITE of winning, that prove PER’s existence wrong. At it’s best, PER is a no-duh statistical analysis and, at it’s worst, which is most of the time, it is a highly flawed and easily deletable construct.

But for some reason the PER has taken the basketball world by storm. I’ve been to many websites who support any argument with PER. An entire website, HEATSPN (also known as ESPN), runs ENTIRELY off of the PER analysis, using it in player ID cards and in fantasy leagues for reference.

The idea has taken root in many places and, for some reason, is successful and accepted. And Hollinger has made a living of providing something presented as raw and true but is subject to unclassified elements.

And his Power Rankings are his worst creation: a monster worthy of destruction. While PERs are mostly ‘correct’ because, in regards to great players, the numbers back them up, the PER completely devalues the non superstars and role players. . .and a TEAM, which is what the Power Rankings look at, is comprised almost entirely of players who provide that which stats can’t describe.

So while a PER can easily be acknowledged or thrown away (being both obvious and inconsequential), a Power Ranking comprised of Hollinger analysis is almost entirely without merit and completely ridiculous. And each year, as different teams come from obscurity or old teams, like the Spurs, continue to remain consistent, the Rankings get crazier and crazier.

Here is a paragraph by paragraph breakdown of Hollinger's Power Rankings. All italics are copied directly from Hollinger's ESPN page for the purposes of argument:

“These rankings are based on a formula I devised, and they are updated every day, automatically.

I created these rankings to give a quick assessment of all 30 teams so far in the season, since sometimes the standings can be misleading in this department.

Scoring Margin

One of my goals was to create a system that told us more about a team’s quality than the standings do.”

Here is the first problem: Standings exist for a reason and they absolutely indicate a team’s quality. The classic expression is ‘a win is a win.’ And in soccer, the common axiom is that ‘true champions/winners find a way to win even when they are playing badly.’

The same applies to basketball. If a team can squeak out 12 wins in 13 games by small margins, it certainly outdoes an 8-6 team that had four blowouts and four close ones in the win column. Because playoff seeding and positioning is not dependant on how someone won what games they won but on how many games they actually won! But Hollinger has an explanation for this:

“So instead of winning percentage, these rankings use points scored and points allowed, which are better indicators of a team’s quality than wins and losses.

This might not sound right at first, but studies have shown scoring margin to be a better predictor of future success than a team’s win-loss record. Thus, scoring margin is a more accurate sign of a team’s quality.

That explains why, for instance, four seasons ago the Spurs ranked ahead of the Mavericks even though they had won nine fewer games — San Antonio’s scoring margin was superior. That ultimately proved to be prophetic, as Dallas lost in the first round of the playoffs while the Spurs won the championship.”

Second problem, especially in regards to the example Hollinger uses here. We talked earlier of the human mind coming into play. The Dallas Mavericks had made a habit of sorts of blowing big games (they still do). 

In 2006, before the year mentioned as Hollinger’s example, the Mavs lost in heartbreaking fashion to the Miami Heat in the NBA Finals: They lost four straight games after winning the first two, something that doesn’t happen often.

In 2007, they were a better regular-season team by nine games over the Spurs. They were actually one of the greatest teams ever winning 67 games (if memory serves). They were insanely good and had the MVP that year playing for them as a star.

But no one expected the Golden State Warriors. I’m sure Hollinger didn’t have them ranked high on his Rankings yet they beat the Mavericks due to two factors: a) the Warriors collection of unflinching, hardcore athletes, mostly cast-offs and troublemakers and b) the Mavericks getting the mental block.

Losing in the first round to an eighth seed was just expected for a team like the Mavs whose reputation was shot from 2006 and had not only a giant chip on their shoulder but a massive monkey on their back. The monkey weighed too much. So, yes, Hollinger’s had the Spurs ranked higher due to stats, and it was fortunate the Spurs won the title that year to help his theory.

However, there was more HUMAN involved in the Spurs run then Hollinger’s numbers. For one, you had a mini-dynasty at work. They had a proven roster of players who knew how to handle what they had to handle, some of whom had three shiny rings on their fingers. Plus, their path to the championship was rather weak.

Another anomaly impossible to predict was one of the few times LeBron James showed any zest and clutch in the playoffs and basically single-handedly knocked off the excellent Pistons in the Eastern Conference Finals. What you had was two superior teams knocked off by freak chance. And San Antonio could have played the corpse of George Gervin (is he dead?), and they’d have beaten the Cavs in 2007.

My point is, there were factors BEYOND analysis that determined the Spurs route to a ring. Their win margin had nothing to do with fate and the fallibility of man. Were the Spurs good. . .absolutely. . .but if you’re going to take the credit for the ‘success’ of a Power Ranking, then you have to realize its limitations by accepting the unexpected which the Power Rankings could not predict.

I highly doubt the Power Rankings had the Warriors in round two of the playoffs, forcing the Jazz to six games, or the depleted Cavs in the Finals. With the Spurs, Hollinger’s Rankings were due to coincidence, not brilliant statistical analysis.

The same can go for 2009, another enigma year, in which the third-seeded Orlando Magic knocked off the second seed Celtics and the first seed Cavs on their way to the NBA Finals. Everyone, including Prince James himself, had a ticket to the Finals punched for LeBron and co to meet the Lakers.

The Magic, a team far lower team on the Rankings due to closer games and less blowouts, unlike the Cavs who stomped on competition like giants, managed to take out the higher ranked opponent(s). (Unfortunately, I do admit I can’t find the old Power Rankings in their entirety on the intertubes so I am going off of what I assume based on seeing Hollinger’s PR every week).

And to counter with the Magic again, the Magic annihilated everyone in the second half of the 2010 season, going 33-8, or something, and winning a ton of games in double digits. They got as far as the conference finals, when a tough-minded, defense heavy team, that usually didn’t win by large margins, grinded the Magic down.

Recent Performance 

Another key variable in the formula is recent performance, which I included for two reasons.

First, it stands to reason that more recent games are more valid indicators of how strong a team is currently.

Second, I wanted these rankings to follow the model of Marc Stein’s “human” power rankings, on the site each Monday, in which a team’s recent play is a huge factor.

To accomplish this, I weigh a team’s full-season results by two-thirds and its most recent games by another one-third, so the overall ranking gives greater weight to recent games.

You’re probably wondering at this point what I mean by “recent.” It varies depending on where we are in the season.

For the first 40 games of the season, it means a team’s past 10 games.

From that point forward, however, it means the most recent 25 percent of a team’s schedule. The net result is that, after the first 40 games, a team’s most recent 25 percent of its schedule will account for 40 percent of its ranking.”

My problem here is that the Rankings become even more flawed by using recent results to complement overall numbers. How can this not be achieved by simply looking at a record? If a team started 10-1 and 10 games later is 15-6 and have lost four straight, it is pretty evident they are going through a rough patch. At that point, it is understandable that particular team would drop in the rankings because other teams, perhaps a 10-1 team that’s now 19-3, is playing better.

I see what Hollinger WANTS to say: Over time, the equation he’s put together will show who is truly superior. But, as I explained earlier, a win or lose in a Game 7 by one point or 70 points is still a win/loss, and any player, PER high or low, can make these Power Rankings completely pointless (and mostly does). It’s funny that something organized so completely on the massive quantity of numbers can be destroyed by one, single integer. 

But, thank god for. . .

Caveats 

Since this is an entirely automated ranking, you’ll notice certain “human” factors missing.

It doesn’t know which players are about to come back from injury or which teams have been playing without their best players for the past 10 games.

Along the same lines, it doesn’t take into account injuries, trades, controversial calls or any other variables — just the scores, please.

Nonetheless, it can be very useful because it allows us to see what the landscape looks like when we remove our usual filters.

We hope you enjoy our daily power rankings.”

Well, to address the last part: I don’t, if that hasn’t been clear already. But back to the beginning. I’ve been making this point all along. Example: Suns-Spurs 2007 Western Conference Finals (poor Suns). The Suns had pretty much clinched Game 4 to tie the series at two but due to a crazy act of oddness by Robert Horry, a career knucklehead, that ended up being a BONUS, and a strange league rule (not even an in-game rule), Phoenix was robbed of two of its best players for Game 5.

They of course lost, though they actually had the lead and were undermanned for almost the entire Game 5. So by asking, ‘just the scores’ please, the Power Rankings become instantly unbelievable and inaccurate. If you go off scores only then you are purposefully looking AWAY from how the game works. Why? Because almost all of basketball, as I’ve said ad nauseum here, is NOT SOMETHING YOU CAN CALCULATE.

And this leads me back to the beginning of this ridiculously long rant: the ACCEPTANCE of this method. I’m not sure how Hollinger did it but he managed to make himself a household name and figurehead of an entire movement that is, at its core, flawed and the antithesis of what basketball is all about.

And while there are plenty of detractors like myself (many who feel their ‘team’ should be No. 1, or whatever), this system of thinking has gained steam and taken hold of a basketball nation.

In the end, as Mulder said, if we rely too much on the ‘foreseen’ we’ll be shocked, like in 2007, 2009 and 2010, by the reality of the whole thing. Hollinger’s been proven wrong by time and the game itself. . .am I the only one who has realized this?