Advanced statistical analysis has revolutionized the world of sports. In "Those Guys Have All the Fun," the recently published oral history of ESPN, John Walsh, head of production for ESPN, laments that many of the best business candidates they try to recruit from Yale and MIT are being snatched up by professional sports teams.
This is often called the "Moneyball revolution" (named after Michael Lewis' 2003 best-selling book about the Oakland A's).
Personally, I've always thought this was a good thing. Baseball thinking, along with thinking in most of the sports world, is antiqued and largely subjective. Major League Baseball still uses Henry Chadwick's error statistic, despite the fact that it might be the most erroneous measurement in all of sports.
At its heart, the "statistical revolution" in sports is not about statistics; it's about asking questions of accepted knowledge and then searching for answers.
It's about finding value by exploiting the intellectual shortcomings of others. It's about gaining knowledge—and thus an advantage—that others lack, or haven't found yet.
As the statistical revolution has evolved, it has encountered one glaring shortcoming—an over reliance on normative truth. What do I mean?
The probability of most events is always changing. The mistake we often make is that we perceive them as an exact probability (the way we do a roll of the dice, or picking cards from a deck).
But this is a lie. Nassim Taleb calls this the ludic fallacy in his book, "The Black Swan," and it is prevalent in the world of sports.
Tim Tebow's completion percent on every play is not 44.8—his average for the year. It is always in flux.
Tebow's chances of completing a pass are dependent on a myriad of factors, the smallest of which can greatly effect the outcome. Anyone who watched "Jurrassic Park" (or, for you Ashton Kutcher fans, "The Butterfly Effect") knows this is chaos theory. And, yet, we so often forget its teaching: Small changes can have large impacts on complex events.
Consider what has to happen for Tebow to complete a pass. The play call is dependent on the down, distance and the game situation. The play has to get into the huddle with a fair amount of time on the play clock.
Tebow comes to the line of scrimmage where the defense, just like the offense, is changing its approach based on the down, distance and game situation. The defense has to adjust to the Broncos' offensive alignment.
When the ball is snapped, the Broncos' wide receivers have to correctly run their routes. The defensive backs judge where the receivers are headed and adjust to that.
Tebow adjusts to what the defense is giving him. His offensive line has to block the oncoming rush. If it's a blitz, Tebow has to hurry his throw and his running back has to stay in to block, rather than run a passing route.
Then, the pass—if a receiver is open and the defensive line hasn't prevented it—is thrown.
Finally, we are at the outcome we are focused on—Tebow's ability to complete the pass.
The exact probability of that outcome is now impossible to measure. The exact chance of success is never exactly what it was going in, or what the average of the sum total of events afterward is.
The big lie here is the use of an average to evaluate complex situations. The majority of outcomes are not fixed probabilities the way a roll of the dice is.
Small things make a large difference in the outcome of the event. A running back missing a block can foil a perfect play call. A rainy day can affect Tebow's grip on the ball, or the receivers' ability to make a clean cut into their route.
Avoiding the ludic fallacy is one of the biggest challenges sports fans and statistically inclined people face. It is arguably one of the largest misconceptions most people face.
However, I deeply hope it might also inspire people the way it did objective thinkers in baseball to challenge the darkness of the unknown—to ask new questions and relentlessly pursue a new and better understanding.
Perhaps, in some small way that will have a large impact, Tebow will help us get there.