The Best BCS Conference As Determined By Science

Scientizzle FoShizzleContributor ISeptember 4, 2008

An interesting article was published on Bleacher Report a little while back regarding the now-constant debate among college football fans regarding the relative strength of the major conferences.

In this article, the author (BabyTate) asks which league—SEC, Pac-10, Big Ten, or Big 12—has placed a greater percentage of teams in the final top 25 polls from 2000 to 2007.  It's a reasonable and simple analysis, and here's what it reveals.

"The Pac-10 placed 30 percent of its teams in the Top 25, the Big 12 notched 32 percent, the Big Ten 33 percent, and the SEC led with 41 percent."

Interesting stuff—but I think it lacks nuance and can benefit from a more in-depth exploration.  I'd argue that the position in the polls is also important, as BabyTate's analysis equates every No. 1 with every No. 25.  I propose that a truer measure would be to analyze the relative position as well.
Also, the teams of the 2000 season and the teams of last year are quite a bit removed.  I think temporal trends are worth evaluating.  Position can be measured & weighted rather simply.
Here's how:

* No. 1 in the polls gets 25 points (and so on, to No. 25 earning one point).

* The maximum points available each year in each league depend on the number of teams in the league.  For example, the 10-team Pac-10 can earn up to 205 total possible poll points each year (25+24+...+17+16=205).

* Divide total numbers of earned points by total possible points to determine the percentage that accounts for the weighted top 25 results.  This is the percentage of maximum possible poll points.

Lather.  Rinse.  Repeat.
What's the comparison then?  I'll use the season-ending AP & ESPN/USA Today polls (using the average ranking) from 2000-2007.  Keep in mind, the ACC & Big East are more complicated because they swapped some teams around and changed size in 2004 & 2005...I left them in, though.
After far too much work, here are the results...
From 2000-2007, these leagues earned the shown percentage of possible ranking points.

SEC: 28.5 percent

Big Ten: 22.7 percent

Big 12: 25.5 percent

Pac-10: 22.7 percent

ACC: 16.7 percent

Big East: 19.4 percent

The raw numbers here won't compare nicely with the Bleacher Report analysis, but let's look at the rank order.
His: SEC > Big 10 > Big 12 > Pac-10
Mine: SEC > Big 12 > Pac-10 = Big 10 > Big East > ACC
My paradigm shows the Big 10 & Pac-10 in a dead heat and the Big 12 undervalued by the Bleacher Report analysis.  The SEC clearly comes out on top.
How about over time?
For the last four years, the SEC has dominated in this metric—and it's a conference that appears to be trending upward.  I'd buy this, based on the reality that there are a lot of great teams in that conference every year.  Currently the league also boasts a Who's Who? list of college coaches and the last two BCS champs.
The Pac-10 has been fairly steady: never the worst and never the best.  Yearly, the recent dominance of USC has been accompanied, almost invariably, by two teams in the No. 10-20 range.  I would bet good money on another year of the same...
It's easy to see how the Big East suffered—notice the fat dip 2003-2005—during the moves of Miami, Virginia Tech, and Boston College to the ACC.  ( didn't seem to help the ACC much, did it?)
The Big East has regained some strength on the backs of strong West Virginia and Louisville squads, the establishment of South Florida and Rutgers as legit top 25 teams, and the recent success of Cincinnati and UConn, giving the Big East some (unexpectedly good) clout.
The ACC used to be the Florida State show.  Now the Seminoles aren't even good.  Virginia Tech has been the only program with any semblance of consistency and national recognition since the realignment.  Recent growth by Wake Forest notwithstanding, until Miami and FSU get back to top dog status, the ACC may continue to slide.
The Big 12 slid for seven straight years but had a good year in 2007.  Will they continue to bounce back?  I think that may hinge on the reestablishment of traditional North Division power Nebraska.  Texas & Oklahoma have been carrying the conference while A&M underachieved, K-State regressed, and Colorado's program was rocked by scandal after scandal.
Also in play: Can Texas Tech finally take that last step up in performance (that is, win all those games they're supposed to win) to become a truly elite team, rather than an inconsistent-but-very-good team with ridonkulous PlayStation offense numbers?
The performance of the Big Ten has been much-maligned on teh intarwebs the last couple years.  I think what my analysis indicates is that the conference has bounced up and down in relative strength.  Last-last-first-first-fourth-second-third-fourth.  Wha?  What's interesting is the difficulty in divining any sort of trend here...
So, let's let the computer do it.

These are the best-fit linear trends of each conference.  What jumps out to me is that four of six show downward slopes, and the Big Ten isn't one of them!

Now admittedly, this isn't a large sample set—eight years of data in an imperfect metric won't exactly help me earn that Ph.D. I'm supposed to be working on—but perhaps the Big Ten is getting a bum rap?  Maybe 2002/2003 were plus-side aberrations of a generally improving league?

Nah.  I still can't buy it.  Right now, Ohio State is the only team that strikes any fear into non-conference foes (Sep. 13 is gonna be huuuge).  Michigan is sliding, Wisconsin's success seems a little over-inflated, Minnesota should be relegated to the Mid-American Conference, and Iowa has lost its mojo...
...Does this mean I need to go back and collect earlier data for a stronger analysis?  I don't know if that's something I wanna do.
One final iteration of this analysis: What about the non-BCS teams?  The graph below includes the yearly result (and associated trendline) of the fraction of available points that went to teams outside the six BCS conferences.  [This data technically uses a different y-axis (percentage of total poll points), but I left that info off the already-cluttered graph.]
We can see that non-BCS schools have also been able to increasingly chip away at the number of available ranking points for the big boys.  This coincides well with the general feeling that modern college football has much greater parity than ever before.
I recognize the main limitations of this analysis: My data set is the human polls, which are subjective measures of relative team strength that can be shaped by the voters' regional and temporal biases.  However, there aren't many alternatives that have general acceptance and convenient measures.