Putting MLB Experts' Annual Predictions to the Test

Zachary D. RymerMLB Lead WriterApril 1, 2013

How has ESPN's Buster Olney done with his preseason predictions recently?
How has ESPN's Buster Olney done with his preseason predictions recently?Don McPeak-USA TODAY Sports

If you want to know how a given Major League Baseball season is going to pan out, your only practical option at this point of the year is to listen to the experts.

But be warned: When you listen to the experts, you do so at your own peril.

If you're wondering which experts you can rely on the most, that's the tricky part. There are plenty of experts out there making predictions year after year, but they're not exactly prophets making prophecies.

How do I know this?

How else? I took a few hours over the weekend to round up some expert predictions from the last couple years and found that the experts are about as good at predicting the future as the rest of us.



Picking Divisions

When it comes to preseason MLB predictions, it's all about picking division winners. That's what fans care about the most because...well, shoot, what else matters?

Picking division winners is a game that experts actually have a pretty good chance at winning. For each division they tackle, they essentially have a one-in-five chance of being right (one-in-four for the AL West and one-in-six for the NL Central before this year, but whatever).

But getting all six division winners right? Good luck with that.

The following table shows how many division winners a collection of 20 experts from various websites and publications have gotten right since 2009. I would have gone back further than 2009, but you'd be surprised how hard it is to find preseason expert predictions from 2008 or before. There's also obviously a lot more than 20 experts out there, but they tend to move around a lot and relatively few seem to have been going on record with their picks consistently in the last four years.

That said, you should recognize most of these names.

Expert Affiliation 2009 2010 2011 2012 Total
Jay Jaffe Baseball Prospectus 2 4 1 2 9
John Perrotto Baseball Prospectus 2 2 2 2 8
Jim Caple ESPN 2 3 1 2 8
Jerry Crasnick ESPN 3 1 1 2 7
Pedro Gomez ESPN 2 2 1 3 8
Eric Karabell ESPN 4 1 1 2 8
Keith Law ESPN 1 3 2 2 8
Buster Olney ESPN 4 2 1 1 8
Peter Pascarelli ESPN 5 2 2 4 13
Jayson Stark ESPN 4 2 3 3 12
Jon Heyman Sports Illustrated/CBS Sports 3 1 1 2 7
Albert Chen Sports Illustrated 0 4 2 2 8
Cliff Corcoran Sports Illustrated 2 3 1 2 8
Ted Keith Sports Illustrated 3 3 2 3 11
Joe Lemire Sports Illustrated 3 2 2 3 10
Ben Reiter Sports Illustrated 3 3 2 3 11
Tom Verducci Sports Illustrated 4 2 1 4 11
Tim Brown Yahoo! Sports 3 2 0 1 6
Steve Henson Yahoo! Sports 3 1 2 2 8
Jeff Passan Yahoo! Sports 1 2 2 2 7

Keep in mind that the last column on the right is out of 24. Out of these 20 experts, only two have gotten at least half their division picks right over the last four years: Peter Pascarelli and Jayson Stark.

Take a bow, you two.

Beyond the two of them, you have to hand it to Sports Illustrated for having four guys who have gotten at least 10 division picks right since 2009. As you can tell, that's not a bad success rate for a single publication relative to the others.

If you're sitting there pondering why there's not even one expert out of these 20 who has blown away the competition with his picks, the simplest explanation also happens to be the best explanation.

Experts make their picks based on what should happen, and anybody who has been watching baseball for long enough will know that the things that should happen don't always happen. Baseball's not as predictable as, say, basketball.

Case in point: Remember the 2011 Boston Red Sox? Remember how they added Carl Crawford and Adrian Gonzalez over the winter and were thus primed to go 162-0 and win the World Series?

Yeah, that didn't pan out. The Red Sox had a spike strip put in their path by the baseball gods heading into September and they stumbled accordingly. When they finished the month with a 7-20 record, many expert predictions died a painful death.

Of our 20 experts, 18 had the Red Sox tabbed to win the AL East in 2011. The only two who didn't: John Perrotto of Baseball Prospectus and Albert Chen of Sports Illustrated. Picking the New York Yankees to win the AL East was going against the grain at the time, but they had the guts to do it.

But then you have division winners that nobody sees coming. Nobody had the Arizona Diamondbacks winning the NL West in 2011, and nobody had the Oakland A's winning the AL West last year. Much to the chagrin of the experts, these things happen.

So don't expect any expert to be right with all six of his division picks in a given year. Based on our data, the best anybody's going to be able to do is four or five correct picks, not six. Even three out of six ain't bad next to the general success rate.

If you think that's unspectacular, just wait until you get a load at how the experts have done picking World Series teams in the last four years.


Picking the World Series

Experts stand a chance when picking division winners, but they stand far less of a chance when picking World Series teams. There's a pretty big difference between picking winners out of six small hats and picking winners out of two huge hats.

Since the odds of being right are so slim, you won't be surprised to hear that 18 of our experts—I was unable to find preseason World Series picks from Jay Jaffe, John Perrotto and the rest of the Baseball Prospectus illuminati—haven't had much luck picking World Series winners over the last four years.

Expert Affiliation AL Champ NL Champ WS Champ
Jim Caple ESPN 0 2 1
Jerry Crasnick ESPN 0 0 0
Pedro Gomez ESPN 0 1 0
Eric Karabell ESPN 0 0 0
Keith Law ESPN 0 0 0
Buster Olney ESPN 0 1 0
Peter Pascarelli ESPN 1 0 0
Jayson Stark ESPN 1 1 0
Jon Heyman Sports Illustrated/CBS Sports 2 0 0
Albert Chen Sports Illustrated 0 1 0
Cliff Corcoran Sports Illustrated 0 1 0
Ted Keith Sports Illustrated 1 0 1
Joe Lemire Sports Illustrated 0 1 0
Ben Reiter Sports Illustrated 0 0 0
Tom Verducci Sports Illustrated 0 1 0
Tim Brown Yahoo! Sports 0 0 0
Steve Henson Yahoo! Sports 0 0 0
Jeff Passan Yahoo! Sports 0 0 0

Stand and be recognized, Jim Caple and Ted Keith. Out of a group of 18 of the finest baseball experts the expert community has to offer, you're the only ones who have picked a World Series winner within the last four years.

Caple, who's gotten only eight division picks right in the last four years, nailed the San Francisco Giants winning the National League pennant and the World Series in 2010.

Caple was the only ESPN expert to do so, as his colleagues went mainly for the Red Sox, Yankees and Philadelphia Phillies. The World Series favorite among NL West teams was actually the Colorado Rockies, for reasons that are now lost to us.

As for Keith, the pick he nailed was the Yankees winning the World Series in 2009. The only other Sports Illustrated expert who had them winning it was Melissa Segura, who is not included in our study because she disappeared from SI's panel after 2010.

It's a bit odd, in retrospect, that more experts didn't see the Yankees winning the World Series in '09. The club may have missed out on the playoffs for the first time in a bajillion years in 2008, but adding CC Sabathia, Mark Teixeira, A.J. Burnett and Nick Swisher made for a darn good offseason. The experts didn't flock to them like they eventually flocked to the 2011 Red Sox. 

Feel free to draw your own conclusions as to why, but I'm guessing the experts shied away from the Yankees because, shoot, picking them is just too easy, isn't it?

Either way, the '09 Yankees look like the most obvious preseason World Series favorite from the last four years. Caple nailed the Giants in 2010, but that was certainly a bold pick at the time. In light of the hype surrounding the Red Sox and Phillies at the start of the season, the St. Louis Cardinals were a surprise champion in 2011. The Giants weren't too far of a long shot last year, but they didn't stand out as an obvious World Series favorite.

But, again, it's not really fair to bash the experts for their collective hopelessness in picking World Series winners over the last four years. Predicting what's going to happen in the regular season is hard enough, but predicting what's going to happen both in the regular season and in the postseason?

That's a whole 'nother kind of difficult. That's like trying to predict a rainy Friday six months down the line and the accompanying ticket sales at the local cinema.


The Lesson Learned?

In a nutshell: Don't trust the experts. Or, at the very least, think twice before betting the farm on their predictions.

It sounds mean, but I'm sure they'd all say the same thing. They make predictions a) because they're obligated to do so and b) because making predictions is fun for all involved parties. We're not talking about anything sacred here.

One guy who understands this better than most is Ken Rosenthal of FoxSports.com. I didn't include him in the study because I couldn't find his division picks from either 2009 or 2010, but I did find his other picks. They're not pretty, as he had the Chicago White Sox winning the World Series in 2010, got one out of six divisions in 2011 and improved to only three out of six last year.

This year, Rosenthal introduced his 2013 predictions by writing: "There’s no getting around it—my predictions suck."

Don't beat yourself up, Ken. Based on what we've seen, it's highly, highly unlikely that you're the only expert who feels that way about his annual predictions.

It's the nature of the beast. Predictions are dressed up as gospel, but they're more like lottery numbers: wild guesses that are very likely to "pluh" in the end.



Baseball Prospectus: 2009, 2010, 2011, 2012

CBS Sports: 2012

ESPN: 2009, 2010, 2011, 2012

Sports Illustrated: 2009, 2010, 2011, 2012

Yahoo! Sports: 2009, 2010, 2011, 2012


If you want to talk baseball, hit me up on Twitter. 

Follow zachrymer on Twitter