Beane Counting: How To Grade an MLB General Manager

(Photo by Justin Sullivan/Getty Images)

Although many refer to baseball's current generation as the "steroid era," far from the only changes in the game can be traced to medically enhanced physiques. Pitcher wins, losses, and saves, batting average, RBI, errors and bunting all seem to be on the way out in favor of adjusted ERA, BABIP, VORP, range factors, and pitch counts, all leading to a newer era in baseball influenced by the work of sabermetrics—trying to measure objectively what leads to wins and losses, challenging the conventional wisdom, and often upending it.

The book Moneyball and the success of small-market clubs like the Oakland A's in the early 2000s had a lot to do with this changing of the guard—and a consequence of that is the fact that general managers are in the public eye in a way not seen before in baseball history.

For example, nearly every baseball fan would associate Billy Beane with the Athletics more quickly than most of the current A's on the roster, but few would know offhand the executive of great teams of the past. Does anyone associate Bob Howsam with the Big Red Machine?

Because of this, a great irony of this sabermetric era has erupted—great strides have been made in measuring players objectively, but the man who has assembled that team gets a free pass when it comes to such analysis. In my experience, arguments among my peers about the best GMs in baseball tend to consist of hyperbolus statements like "X is the worst GM ever" or turn one bad transaction into the basis of failure, such as "He traded X, Y, and Z for W! That's an awful trade."

Discussion like this seems to get us no closer to an answer. Some articles have made an attempt at it, but the flaws in each seem to undermine the results.

David Gassko tried to do this in 2005 with his article Ranking the General Managers at The Hardball Times—basing his rankings on three factors: how well a team is built for its park, how much the team wins compared to its payroll, and how a team gained or lost at the trade deadline.

This only was done for the 2004 season, meaning that the basis for all of these factors was only that particular season—though that was the only goal of the article, so it is understandable.

However, the problems in the formula are significant. Building a team for a home park may not even be a positive factor at all—I can understand why it was included, as it may seem wise, but when the measurement basis is just having a better home team than road team, it can be seen as a detriment just as easily as a positive.

The midseason transactions idea is interesting, and has some merit, but both of these factors are taken into consideration as greatly as the "bang for your buck" of the team. That would seem to take precedent over either of the other factors—especially the home field—and that would cause significant problems in accepting his results.

Forbes published a study that measured the best GMs in sports in 2007, with the baseball GMs ranked here, which factored improvement on a predecessor and payroll relative to the league.

Aside from the problems of making this a sports-wide exercise without adjustments to the winning/payroll conditions of each specific league, the idea of comparing a GM only to a predecessor has its problems. Any GM with an historically awful predecessor will rank highly even with only a mediocre job on his part, and any GM with an historically great predecessor will only rank as mediocre even if he can maintain the team's success.

Also, winning is factored twice as much as payroll—an idea I don't object to in theory, but there is no reasoning for why this proportion was decided.

Among the factors seen, the only variables that can truly be used are the success of the team and the relative payroll. These factors also allow for comparison between different years, since the payrolls can be adjusted.

This idea is common in subjective analysis—another common statement in measuring GMs is "He won X games with a $Y million payroll in year Z." However, without a baseline for exactly how many games a certain payroll should have won in a certain year, how can any conclusions be drawn from it? Without that question answered, there is no logical starting point.

In order to get this baseline, I made a database of team payrolls and Pythagorean winning percentages for every team I had payroll information available for (I used USA Today's database, found here).

To adjust for the different average payrolls of each year, I simply found the Z-score of each payroll—figuring out how many standard deviations it was away from the mean spending of that season. By running a regression on those variables, an equation that shows what should be expected from a team at a certain payroll:

Y = 0.0263293618645785(X) + 0.500400325761534

This equation shows for every standard deviation the payroll changes, it should increase or decrease team winning percentage by about .026 (in terms of a 162-game season, that's a little over four wins.) The equation has a correlation coefficient of about 0.17 (which is significant with 600 team seasons worth of data), but indicates that having a large payroll is far from enough to win games (sorry, critics of Brian Cashman).

The middle 95 percent of payrolls fall between a Z-score of -2 and 2, which, according to the formula, indicates that an average GM should win 72.5 games at an extremely low payroll and 89.6 games at a relatively high payroll. (The only team to break these constraints on a regular basis are the Yankees.)

This is an interesting conclusion—73 wins for a team with minimal financial resources seems like a better than average job, in the same way that a team with 90 wins with a huge payroll seems like a bit of a disappointment, but this reinforces the correlation coefficient's determination that money is somewhat overrated in baseball.

You're currently logged into Facebook

Do you want to share your Bleacher Report activity on Facebook?