The Effects of the Officials in Super Bowl XL Using Win Probabilities

Pop quiz: Who said regarding the referees, “I felt they were cheating us,” and “The way the refs were going, I wouldn’t have trusted them in overtime,” and that they were “[taking] the game away from us”?

If you answered anyone from the Seattle Seahawks, try again. The answer is former Pittsburgh Steelers linebacker Joey Porter, who said that after the officials overturned Troy Polamalu’s fourth-quarter interception against Indianapolis in the AFC Divisional Round.

The NFL must have heard his message.

Three weeks later, the Seahawks were not only facing the Steelers in Super Bowl XL, but also Bill Leavy and the rest of the officiating crew. In the biggest game on the biggest stage, the officials made so many questionable calls that benefited Pittsburgh, all they were missing was a Steelers’ helmet.

Seattle was called for seven penalties for 70 yards, and Pittsburgh was called three times for 20 yards.

Said Seahawks coach Mike Holmgren, “I didn’t know we were going to have to play the guys in the striped shirts as well.” Jason Whitlock wrote about the officials, “Am I the only one who would like to hear them defend their incompetence?” Michael Smith said that the officials were “giving [Pittsburgh] the game.”

By all means, Seattle was robbed of the Super Bowl, or at least a shot of winning.

In order to quantify the effect of the officials on the game, I turned to the win probability calculator on Advanced NFL Stats, which I previously used to figure out when to go for the two-point conversion.

By looking at the win probabilities of Seattle prior to and after each of the seven penalties handed to them, I could see exactly how much the Seahawks’ penalties hurt them.

Before I get to Seattle, I’ll show the effects of Pittsburgh’s three penalties as a baseline. PRE shows the Steelers’ situation before the penalty occurred, and POST shows what happened after the penalty.

Expected points come from the win probability calculator, and end result is the final outcome of the drive in which the penalty occurred.

Pittsburgh’s penalties lost them an average of 2.7 percentage points on their win probability and less than half an expected point.

How does compare to Seattle? A chart showing the seven penalties called on the Seahawks and the change in their win probability is below, with a description of each penalty and its effect below that.

First penalty—first quarter, 5:53 left, offensive holding: After an 18-yard pass to Darrell Jackson, Seattle was called for holding. The Seahawks went from first-and-10 from the PIT 23 to third-and-16 at their 49. They were well in field goal range prior to the holding call, but had to settle for a punt after failing to convert the long third-down try.

Second penalty—first quarter, 2:00 left, offensive pass interference: Darrell Jackson caught a 16-yard pass in the end zone for an apparent touchdown. But the back judge—Bob Waggoner, who is a Pittsburgh native—called a late penalty on Jackson that pushed the Seahawks back to a 1st-and-20 on the PIT 26.

Seattle kicked a field goal, but if the touchdown to Jackson had stood, the Seahawks would have been driving for the game-tying touchdown on their final possession (which ended on an incompletion on 4th-and-7 from the PIT 23 with eight seconds left in the game).

Third penalty—second quarter, 14:44 left, offensive holding: Seattle returned the Steelers’ third punt of the game 34 yards to Pittsburgh’s 46-yard-line, but holding was enforced at the SEA 35.

The Seahawks took possession at their own 25 and ended up punting at the PIT 47 on fourth-and-two. The 29 yards Seattle lost would have put them at Pittsburgh’s 17 with a chip-shot field goal if there had been no penalty.

Fourth penalty—second quarter, 1:46 left, offensive holding: The Seahawks committed a holding penalty on the kickoff return following Pittsburgh’s first touchdown, which lost Seattle only 10 yards.

Though that may not have seemed like a big difference—what with just 1:46 left and the ball on Seattle’s 27 after the penalty—the Seahawks managed to drive down to Pittsburgh’s 36 and missed a 54-yard field goal to end the half.

Without the holding call, the field goal would have been 44 yards; then-kicker Josh Brown has made six percent more 40-yard field goals than 50-yarders in his career.

Fifth penalty—third quarter, 4:37 left, false start: The most insignificant penalty by Seattle, the Seahawks went three-and-out and punted. Still, Seattle had a 13-yard pass play on third-and-15, which would have been a first down if not for the false start penalty.

Sixth penalty—fourth quarter, 12:08 left, offensive holding: This, on the other hand, was huge. After an 18-yard pass to Jerramy Stevens that ended up on the Steelers’ one-yard-line, the officials called tackle Sean Locklear for holding. The play dropped the Seahawks to the PIT 29 facing a first-and-20, and their win probability fell 17 points.

Two plays later, on third-and-18, Matt Hasselbeck was intercepted by Ike Taylor. Seven points came off the board, and the Seahawks ended with a turnover just outside the red zone.

Seventh penalty—fourth quarter, 10:46 left, low block: On the interception return, Hasselbeck was called for a 15-yard low block penalty. The Director of Officiating for the NFL—Mike Pereira—even said that “the call was not correct” and “should not have been made.”

Nevertheless, the Steelers got the ball on their 44-yard-line and scored on an Antwaan Randle El touchdown pass to Hines Ward.

Although the argument could be made that the Steelers would not have attempted such a trick play on their own 42 (the play occurred 15 yards ahead, on the SEA 43), Hines Ward was so far ahead of his defender that he would have run the extra 15 yards to the end zone.

Adding up all of the effects on the score of each penalty—an extra three points for the first penalty, four for the second call, three for the third, none for the fourth or fifth (maybe Brown misses the 44-yarder), seven for the sixth, and none for the last call—the Seahawks effectively lost 17 or 20 points due to penalties alone.

That would have made the final score 27-20 in favor of Seattle.

The Steelers, on the other hand, lost no points from penalties—two false start calls came on the same three-and-out (and we don’t know if Pittsburgh would go another 70 or 80 yards to score a touchdown if there were no false starts called), and they scored a touchdown on the drive in which a pass interference call occurred.

Would the Seahawks have won Super Bowl XL without the detriments of their seven penalties?

It appears so.