Everyone loves to grade drafts, but not everyone does it well. It's easy to look at player production and slap a letter grade on a general manager, but unless basic rules are followed, the end result is alphabet soup.
Anytime a draft grade is given, ask the following questions before taking it seriously.
1. Has enough time gone by?
Draft grades given out right after the draft are fun, and they are a good way for an analyst to go on the record about a player, but they don't mean much in the long run. Even a full year later, it can be too early to judge most prospects, especially linemen who can take time to develop.
The earliest you can possibly grade a draft pick is two years out. Three is even better.
2. Did the grader consider draft position?
This rule cannot be overstated. Teams drafting at the top of a draft have a massive advantage. Teams drafting in the back half of the first round are at a supreme disadvantage.
Take this draft review of the Indianapolis Colts done by Pro Football Focus. The Colts had far fewer "draft points" to work with than any team in the league. From 2006-2010, they were working with less than half of "league average" in terms of total draft points.
When PFF assembles a list of "draft flops" for the Colts, notice the numbers next to players' names. They list 12 players who didn't work out, but nine of them were taken after pick No. 160. That means the Colts' "draft failures" were largely guys taken in the fifth round and later.
It's normal for teams to miss on late-round picks. The higher the starting draft position, the more should be expected from a draft.
3. What criteria did the grader use?
Football players are hard to grade. Some positions have stats, but not all of them. Did the grader use games played, "approximate value," the eyeball test or some other metric?
As much as possible, teams should be credited when players actually make the field, especially for late-round picks. If a seventh-round player makes the team a couple of times and sees some action, he's had value and the pick was a good one.
Make sure the grader is actually using an objective standard.
4. Did the grader consider context?
The real value of draft pick isn't always in how well he did, but also in how well other players did around him. Imagine a team drafted a two-time Pro Bowl guard, but passed up the chance to grab a Hall of Fame quarterback. The pick of the guard would have to be considered a mistake.
Grading drafts should always be done on a curve. General managers can look great if they get to pick from several different All-Pro players, but some years, the draft just stinks.
Pieces like the Pro Football Focus draft grades are fun, but they lack the overall context necessary to be be truly useful. It's impossible to judge one draft in a vacuum. Drafts can only be judged after several years, side by side with all teams compared, draft position considered and player value measured accurately.
Without those components, you can ignore the letter grades—all you have is a heaping bowl of Subtraction Soup.