Jacoby Ellsbury: The Limitations Of Defensive Metrics

During the course of the last week-to-ten-days, Red Sox Nation has been deluged by a succession of articles detailing Jacoby Ellsbury’s defensive inadequacies in center field.

In the aftermath of the team signing CF Mike Cameron and RHP John Lackey, there has been rampant speculation that the ballclub would ship Ellsbury and RHP Clay Buchholz to the San Diego Padres for first baseman Adrian Gonzalez.

The Sox appear to be laying the groundwork for the team to trade the highly-popular center fielder through the Boston media, which appear to be all-too-willing to serve as an extension of Sox Public Relations Department—sowing the seeds of Ellsbury’s defensive inadequacies.

So it was interesting to learn that baseball fans across the United States voted the young Oregonian as MLB’s “Defensive Player of the Year” in balloting that was conducted on MLB.com at the conclusion of the ‘09 season (the “This Year In Baseball” awards).

And so we again find ourselves at the crossroads of one of the great baseball debates—can baseball officials (as well as fans and pundits) adequately judge a ballplayer’s ability by statistical analysis only (sabermetrics), or by observation only?

Or does it have to be some combination of the two?

It is arguable that the greatest development in baseball over the course of the last three decades has been the proliferation and popularization of new statistical tools for analyzing performance.

The Society of American Baseball Research has been at the forefront of the effort since 1971, and a large number of other individuals have joined the effort in the ensuing years, most particularly during the last twenty years or so.

The sabermetricians, as they are commonly referred to, are constantly tweaking their statistical measures to enhance their ability to analyze and project, and the statistics that they have developed have had varying measures of accuracy and acceptance among the vox populi.

Most devotees of the statistical tools contend that they are the be-all and end-all of performance analysis, but there are some among us (myself included) who feel the statistical approach must be balanced by subjective observation, and that while quantitative analysis is a significant piece of the assessment effort, the numbers alone don’t—and can’t—tell the whole story.

And that brings us to the defensive metrics that have been developed by the sabermetricians—tools that are admittedly flawed and unreliable. These stats are typically based on a subjective observation that provides an objective numerical assignment to a play in the field.

In my opinion, they are almost doomed to failure because it is impossible to perfectly standardize a human observation. Thus, based on the character of the tool, the metric is inherently imperfect.

The most popular (and most widely analyzed/criticized) of these defensive metrics is UZR (Ultimate Zone Rating ). It proposes to determine the number of runs above or below the major league average of a fielder, with respect to runs allowed/prevented: defensive range, strength of arm, ability to turn a double play, and errors combined. The metric was developed by fangraphs.com .

While baseball fans voted Ellsbury the Defensive Player of the Year, the UZR metric instructs us he was the WORST defensive center fielder in the major leagues last season. These determinations are polar opposites—subjectively he is the best, objectively he is the worst.

So the question must be asked: In order for the metric to have some credibility—and to gain a even a small measure of acceptance—doesn’t the statistic have to present at least a nominal relationship to what we are able to observe?

The problem with the defensive metrics is that not only are they inherently flawed by design (the use of the subjective observation), but they often fly in the face of what our subjective observations tell us. It is a conundrum.

An additional problem with these metrics is that they are almost always based on “league average,” and can therefore fluctuate dramatically from year to year based on whether better defenders get injured, get traded, or sign with a team in another league. Their absence from the statistical pool can lower the league-average benchmark…and therefore, a player can go from being “league average” to “better than league average” without ever having improved as a defender.

Again, if you accept the premise that Ellsbury is what he is on defense, you have trouble accepting that his UZR was +3 (3 runs above average) in 2008 and -18.6 (18.6 runs below average) in 2009.

The average baseball fan will NEVER understand how the statistic can vary so wildly—in fact, neither can many sabermetricians. And that underscores a second reason that the metrics are not useful—they do not provide a consistent measure for a skill set that is, essentially, consistent.

(NOTE: For what is is worth, Mike Cameron’s UZR in 2009 was +10.0)

A final note on the problem with defensive metrics:

As I said previously, the defensive ability of baseball players is pretty much a constant—and most sabremetricians will agree with this premise.

In discussions about Yankees shortstop Derek Jeter, fans and pundits alike have widely debated his skill sets as a defender. Most Yankees fans swear by him, while most fans of other teams find him to be below average defensively (some of us believe him to be WELL below average defensively).

UZR shows him to be an average defender, or even a little above average.

Herein lies the problem with UZR and other similar metrics—a problem that will need to be addressed, explained and corrected by the sabermetricians before a metric such as it can ever gain acceptance: If Jeter is Jeter in the field, as assumed, then why did his UZR fluctuated wildly based on the defensive positioning of second baseman Robinson Cano? What does THAT have to do with JETER’S defensive skill set?

It is inarguable that Cano’s position has something to do with the overall infield defense, but it is also inarguable that it has NOTHING to do with the innate ability of Jeter in the field.