Debating the Value of the NBA's Latest Data Breakthrough

Use your ← → (arrow) keys to browse more stories
Debating the Value of the NBA's Latest Data Breakthrough
Richard Rowe/Getty Images

Analytics have been steadily rising to the forefront of the basketball world over the last few seasons, and teams like the Houston Rockets and Toronto Raptors are leading the charge. The success of those two organizations during the 2013-14 season should serve as nice testaments to the value of numbers.

The beauty of analytics, though, doesn't lie in the past. 

Instead, it's all about development. 

The latest breakthrough in the numbers community comes from Kirk Goldsberry. There has been a number of articles on Grantland about his metric, and the public already has access to a research paper he wrote (links to come later on), one breaking down the intent and development of the metric called Expected Possession Value. 

So, what's this all about? What does it tell us about basketball? How much merit does it hold? 

 

Josh Martin, NBA Lead Writer: Nerds of the world, REJOICE! This weekend's MIT Sloan Sports Analytics Conference figures to be abuzz with the newest and most exciting development in NBA analytics: Expected Possession Value (EPV).

I'm guessing EPV isn't entirely new to you guys, though. Kirk Goldsberry, a Harvard professor by day, posted these pieces about EPV and the methodology behind it on Grantland in early February. 

Adam, as one of Bleacher Report's foremost "stat heads" (I use that term with utmost endearment in this case), how would you describe EPV in layman's terms? And what was your initial reaction when the research paper explaining it first crossed your path?

 

Adam Fromal, National NBA Featured Columnist: No parenthetical explanations needed here, because I will always consider that "stat head" term to be a compliment. Numbers never lie, so long as you know how to use them properly, which is a nice segue into the basic explanation of EPV, for those who don't want to get bogged down in the technical and statistical jargon of the paper. 

The best way I can explain it is to bring up the concept of win probability added (WPA) in baseball.

Because MLB action is comprised of more isolated events, each action by a batter or pitcher directly affects the winning percentage of his team. A home run is going to increase the probability of winning a game, but a homer with the bases loaded will affect the outcome even more. Basically, WPA is calculated by giving the player credit for any changes his actions make to the team's winning percentage, based on historical winning percentages of teams in identical situations (scoring margin, number of men on, number of outs and inning). 

EPV is essentially doing the same thing, but breaking the scope into individual possessions and accounting for the continuous flow of a basketball game. Every action directly affects the expected outcome of the team's time with the ball, and the shot at the end of a possession is only a byproduct of those actions.

That's all that's mattered in the past, but we're gaining insight into what leads to that shot. 

To answer your second question, Josh, I was thrilled when I came across the research paper, simply because it introduced one of the first process-based metrics instead of the result-based stats we're accustomed to. Think about the difference between assists per game (result-based) and assist opportunities per game, as shown by NBA.com's SportVU databases. The former relies on the result; the latter tracks how effectively a player gives his teammates chances to score. 

Jim O'Connor-USA TODAY Sports

If I'm passing the ball to you, shouldn't it matter that I got you the rock with an opportunity to score? Let's say that I do that twice, hitting you right between the numbers with a chance for a corner three on back-to-back possessions. The first time, you make the shot. During the second go-round, you clang the attempt of the rim.

Why should I receive less credit for the second situation when I did my job just as well as the first time? 

That seems to be the question at the heart of EPV, and it's one that's needed to be asked for a while now. Fortunately, cameras are now giving us the ability to address it. 

 

Grant Hughes, National NBA Featured Columnist: Adam sets me up perfectly to touch on my favorite part of EPV and its broader import. In this new world we've discovered, he now gets credit for that facilitation—even if I botch the finish.

EPV is about valuing the right things. The smart things. The things that actually matter when trying to make clear judgments on the present and informed predictions on the future. As Adam noted, this is about process and result.

For a while now, we've understood that there are certain spots on the floor that produce the most efficient scoring opportunities. Merely getting the ball to those spots—the corners and restricted area, for example—should be considered a "win" for offenses. If the ball reaches either of those locations, preferably in the hands of a player who can do something with it, the offensive process has succeeded.

What happens from there (a make, a miss, another pass, a turnover) shouldn't matter as much as the fact that the ball made it to a spot with a high expected point yield. We can't totally discount the results of offensive possessions, though. After all, if a guy simply never converts expected value into actual value (hi, Ricky Rubio!), his role in the process needs to be addressed.

Rick Scuteri-USA TODAY Sports

That's a long way of saying we can now plainly see which players are best at putting the ball in places where points are most readily available.

This is a good thing.

There'll definitely be criticism of EPV as it hits the mainstream. The NBA is a results-driven world, and I suspect the vast majority of fans still consider a "good" shot to be one that goes in—and not one that came about as the culmination of a sound offensive process.

How valid is that criticism? And what others should we expect to hear?

 

JM: There's certainly some validity to that criticism. After all, the NBA doesn't award "brownie points" for doing the right thing. The rules are still product-oriented rather than process-oriented, which, frankly, is how it should be. 

But the calculation of EPV takes this into account by differentiating between what the paper characterizes as macrotransitions (e.g., shots, fouls and turnovers) and microtransitions (e.g., movements, screens and other in-possession subtleties). Essentially, microtransitions are all the little things that happen during possessions, while macrotransitions are the actions on which possessions end—and thus, the more inherently "valuable" actions.

Part of the problem with EPV—and this could change, depending on how it's implemented and marketed to the public, assuming it is at some point—is that there's no real explanation as to how an individual accrues his added value. How much value does Player X add by setting solid screens, passing to an open teammate or taking a good shot? On the other end of the spectrum, how much value does Player Y take away if he's setting weak screens, tossing bad passes or hoisting up contested, inefficient looks?

For instance, Table 1 of the paper shows that Chris Paul led all players tracked with an EPV-added (EPVA) of 3.48, but it lacks a breakdown as to where that number comes from on a per-game basis. Knowing CP3's game, we can make an educated guess that it's mostly derived from his passing and driving into the lane, with floaters and jumpers mixed in, but how much do each of those actions add to his value on average?

Derick E. Hingle-USA TODAY Sport

On the flip side, EPV still doesn't do much to quantify an individual's value on defense. Comparatively speaking, a player's offense value is simple to discern. Box score-based metrics—from basics like points, field-goal percentage and free-throw attempts, to more modern derivatives like true shooting percentage, assist percentage, free-throw rate and field-goal percentage by zone—do a pretty good job of illustrating a player's strengths and weaknesses when considered in concert with one another.

But we're still left without much to illustrate an individual's contributions on the defensive end. Blocks and steals speak more to spurious opportunities and, in some cases, risky behavior than they do to sound principles. Measuring how opponents fare at the rim can be useful in discerning the contributions of a paint-patrolling big, and some of Synergy Sports' metrics come in handy when tackling how well one player handles a certain situation over another.

EPV as currently constituted, though, doesn't seem to make that picture any clearer. In the section entitled "Other EPV-derived metrics," the authors hint at its potential utility as a tool for determining what sorts of strategies teams should pursue to coax opposing offenses into less efficient actions. Yet for all the detail with which EPV can supposedly describe one player's offensive contributions, it appears to be rather limited on defense.

Perhaps this speaks to a bigger bone I have to pick with so-called all-in-one metrics like PER, win shares and now EPV. In baseball, all-in-ones are useful because we can already see each individual decision and action so clearly. Thus, the analytics community has devised ways to measure and describe pretty much everything that happens on a baseball diamond and, in turn, combined those into stats like WAR.

Basketball, on the other hand, is free-flowing, without much to discern one action from another on paper unless that action terminates a given possession. We see the product, but are still somewhat blind to the process. EPV has the potential to quantify the process, but does little to qualify it in a way that might actually make the game easier to understand—by breaking the game into its constituent parts and describing each of those in detail.

When thinking about EPV, my mind keeps going back to the same old refrain that Jalen Rose so often spouts on his podcast—you are your skill set. If I'm a general manager, a scout or a coach, I probably won't care what one player's EPVA is compared to that of his peers. Rather, I'd be more curious to see how his EPVA came about, how each of his skills contributed to that number and, from there, determine how that player might fit into my team.

But before I spew all of my thoughts from atop this here soapbox, do you guys agree with my criticisms? Is there some other point or perspective that I'm missing? And is there anything about EPV that grinds your gears as much as those mentioned in my aforementioned blabberings apparently grind mine?

 

AF:  You're stealing my criticisms, Josh, even if that wouldn't give you any benefit in our emailing version of EPV. 

To elaborate on one of your aforementioned points, I simply refuse to believe that all actions changing the value of a possession come via the ball-handler. This is still a team sport, which EPV starts to take into account but doesn't completely roll with. As Lester Freamon infamously said during The Wire, "All the pieces matter." 

When Chris Paul drives to the basket, his EPV goes up because he's such an efficient finisher from mid-range zones and when right around the hoop. But how was that drive completed? Did he break a defender's ankles and burst into the lane during an isolation play? Did he take a solid screen from Blake Griffin and nonchalantly waltz into an area that would help raise the expected value of the possession? 

Mark D. Smith-USA TODAY Sports

If it's the latter, EPV is still going to credit him with changing the value of the possession, and Griffin's contributions go unnoticed. 

Even little things like teammates clearing out help change the makeup of a defense, but that's all being overlooked here. It's also why we're going to see players with possession of the ball forming a bimodal distribution in EPV rankings. They're either going to rank right near the top or right near the bottom—for the most part—because they apparently have the most ability to change the outcome of a possession. 

But do we believe that? Are point guards really the most important offensive players? 

I'd hesitantly say yes, but not to the extent that EPV shows. In the example given by Goldsberry and the rest of the authors, 11 of the 20 players featured in the top and bottom 10s were point guards. 

My other primary concern deals with the following section of the paper: 

Indeed, one major challenge of using data sets as large and rich as optical tracking data is implementing statistical models that are flexible enough to incorporate subtle spatial patterns in players' tendencies while being robust enough to avoid overfitting the occasional abnormal behavior we observe – the Dwight Howard corner 3, for instance. Standard techniques such as regression, analysis of variance, and generalized linear models are ill-suited to these problems. 

There are basically two options here:

  1. Don't worry about the effects of individual players, and look at the results of the NBA as a whole in each situation. 
  2. Take personnel into account (which it sounds like the researchers are doing), and run the risk of inordinately small samples. 

Neither option is particularly positive, though the second is preferable for the very reason explained with the Dwight Howard example. Still, that means it's going to be tough to accept everything at face value. Are you willing to do that, Grant? 

 

GH: I don't know that I believe strongly enough in the perfection of EPV to set myself up as its advocate, but I think it'd be helpful to our discussion if I started out by "arguing" against some of the criticisms you guys have set out. 

First, though, it's critical for us to view EPV as more of a conversation starter than a fully formed, perfect metric. It seems clear to me that the authors are positing this new statistic as a means to discuss basketball on a different level. Moreover, they're embracing a question-drive approach, which is what good research does.

They also don't make any claims as to EPV's current ability to analyze defense. And they also readily admit the tiny and skewed sample sizes that gave rise to the point guard-heavy leaderboard. However, I am immediately skeptical of any purported "catch-all" statistic that doesn't reveal LeBron James to be in the top three. Red flags went up with that omission, for sure.

Issac Baldizon/Getty Images

Josh, I like how you phrased one of your criticisms: "EPV has the potential to quantify the process, but does little to qualify it in a way that might actually make the game easier to understand."

Based on the information we have about EPV right now, I think that's fair. But here's what I'd say if forced to defend the stat: Give it time. We may not yet know how to answer the annoying "So what?" question when it comes to EPV, but the framework is there to eventually provide a response. Again, this is a starting point.

More generally, the questions that seem to be most troublesome for both of you—What else happened? Where were the other players located? Who was involved? Who facilitated or hindered what the ball-handler did?—are the very same ones we could ask about any statistic.

You're right, Adam; Blake Griffin doesn't get credit from EPV for setting the screen that allowed Chris Paul to get to the basket. But what stat does credit Griffin for his assistance there? None that I know of. The fact that EPV has a system in place that could soon give Griffin credit for adding value to that play makes it unique. 

But I think we can all agree that no stat will ever take the place of actually watching the game. 

I also don't think it requires a leap of faith to trust the authors' methodology. They mention that the specific statistical calculations that go into the EPV model are beyond the scope of this particular paper. But they exist, and we'll learn about them eventually.

What we know is the study produces a baseline value against which we can measure individual players. It doesn't particularly bother me that the authors left out specific details on HOW they calculated the baseline, or how they assigned EPV to individual players in relation to that baseline. Maybe that's because I don't view EPV as a finished product.

To answer Adam's last question, no, I wouldn't take EPV at face value. Or, at least I wouldn't feel comfortable making definitive judgments about Chris Paul or Ricky Rubio based solely on what EPV tells us right now. We need to know more about how it's compiled, and how other players influence it. We all agree there.

Noah Graham/Getty Images

But it's a fantastic first step. If you think about it, EPV allows us to quantify (even if it's to an imperfect degree) some of the things we intuitively believe to be true. We think getting into the lane on offense is valuable. We think making the right pass to the right guy at the right time is "good basketball." Now, we have what looks like the first way to measure those things.

Maybe EPV isn't clear enough or exhaustive enough yet. And perhaps it's a long way from being digestible for a broader audience. But it starts the process of talking about basketball in a more thorough way. 

And the process is what matters.

 

JM: The process is, indeed, what matters, Grant. As much as the old curmudgeon within wants to shake his fist, grunt out a few hearty "harumphs" and groan on about how "Back in my day, we played basketball uphill both ways with webbed feet," the point is, this is just the beginning.

What we have to keep in mind, too—and this goes for all stats—is that EPV, like any metric, is most valuable as another tool, another means to an end, rather than an end unto itself. For the most part, EPV could mostly verify what the basketball world already believes and knows about the game from watching it, playing it and poring over the stats that are out there right now.

But there might also be potential for EPV to reveal little things here and there, to hint at possibilities and potential tactics that aren't intuitive or evident to the naked eye. It's those opportunities on the margins, however tiny, that could make EPV a valuable addition to the basketball lexicon.

Bill Baptist/Getty Images

And frankly, that, to me, is the point of the analytics "revolution" about which so many people get their knickers in a twist, one way or another. Teams like Daryl Morey's Houston Rockets and the ghost-haunted Toronto Raptors aren't turning the game on its ear so much as they're nudging the envelope in the direction it was already headed. Before, coaches, players, GMs and scouts alike could infer that shooting more three-pointers or rotating one way or another on defense was a good idea.

Now, even the most risk-averse basketball minds can turn to "big data" to help them predict with even slightly greater certainty how things might turn out if a certain player is thrown into the mix or the mix itself is shaken up strategically. 

 

So, commenters, let us know. How do you see EPV fitting into the mix of basketball analytics at this point?

Load More Stories

Follow B/R on Facebook

NBA

Subscribe Now

We will never share your email address

Thanks for signing up.