Often fans and analysts use "the eye test" as a way of justifying their own unfounded biases. Stats don't replace a solid understanding of the game, but they tend to root out falsehoods.
This week's advanced stat is among the more controversial I'll cover. DVOA is the primary stat of the FootballOutsiders. It stands for Defense Adjusted Value Over Average. It creates a per-play percentage of how much better or worse a player or team is than average. The stat is expressed as a percentile with zero percent serving as a baseline: a player over zero on offense is more valuable than average; a player with a negative DVOA number is worse than average.
DVOA can apply to teams, units or players. It is an efficiency stat, not a volume stat. It indicates how well a team or player performed, rather than how much they performed. There are other numbers that better indicate volume.
There are objections to DVOA. It's complicated. It's a "black box" stat, which means that no one can calculate it thanks to the Outsiders' use of a proprietary formula. It also relies on game-charting data, which can be inconsistent.
The benefits, however, far outweigh the objections. What makes DVOA useful is the fact that it accounts for the strength of the opponent. In a short season, traditional statistics are often heavily influenced by the level of competition a team or player faces. DVOA attempts to account for that.
DVOA for units is also vastly superior to traditional NFL rankings based on yards. Yard-based rankings should never be used for serious analysis. They ignore context entirely. Teams that trail in games will typically post higher passing yard totals than teams that are winning. Good teams will face inflated pass totals and suppressed run totals. Yard-based rankings have no place or value in modern football.
The Jaguars' defense in 2011 is a great example of this. Traditional metrics like yards or even points don't tell the full story of how well they played. Football is a situational game, and context drives many results.
For example, the Jaguars fielded one of the league's worst offensive units. They routinely put the Jacksonville defense in terrible positions. They had the fewest yards per drive and, as a consequence, the Jaguars' defense had one of the worst starting field positions in football. Invariably, this is going to lead to inflated point totals.
Additionally, the Jaguars were often playing from behind which leads to elevated run yardage, even when the defense is generally playing well. The Jags were ninth in run yards allowed using the official NFL measurement by yards. DVOA says they were a bit better than that, ranking them fifth.
DVOA also takes into account the ability of the opposition. By raw numbers, the Texans had a better defense than the Jaguars. However, the Jaguars faced tougher offenses. To illustrate that fact, Jacksonville faced the Texans' offense twice and the Texans got the Jags' offense twice. When you account for variance in schedule between teams, you get a truer picture of overall strength.
What's really impressive, however, is how quickly the Jags jumped up the rankings.
They were 32nd in defensive DVOA in 2010, 21.2% worse than average defense on every play. Last season, they were 6.6% better than average. That kind of swing can be hard to sustain from year to year, but if the Jaguars offense can improve at all, the defense can regress slightly without serious consequences to the team's overall performance.