Pilloried for the decisions they give and castigated for the ones they don't, referees do not have an easy job.
These days it's even harder, as technology allows their every decision to be dissected and discussed by the watching public.
However, some calls are just so bad, they deserve to be pulled apart time and time again.
Over the following slides, we look at 20 controversial refereeing decisions. There's "ghost goals," handballs, feigned injuries and, of course, Graham Poll's three yellow cards at the 2006 World Cup.
Organised in chronological order, this is not an all-encompassing list, so feel free to add your own refereeing gaffes in the comments section below.
The scores were level at 2-2, during extra time in the 1966 World Cup Final between England and West Germany when referee Gottfried Dienst allowed Geoff Hurst's second goal of the game to stand.
It was a questionable call as no-one could tell whether Hurst's effort had actually crossed the line. However, England were awarded the goal.
No-one could dispute Hurst's third, which he put away after 120 minutes, but the goal that gave England the lead will always be shrouded in controversy.
Referee Clive Thomas caused outrage at the 1978 World Cup when he blew the whistle to signal the end of the match, between a corner being taken by Brazil and Zico heading the ball home.
The Group 3 game was just eight seconds into extra time when Thomas made the call and Brazil were denied what would have been their winning goal against Sweden. The tie ended 1-1.
In 1982, France were taking on West Germany in the semi-finals of the World Cup.
The game will always be remembered for a particularly brutal challenge that left Les Bleus defender Patrick Battiston unconscious, missing three teeth and with damaged vertebrae. The former Saint-Etienne and Bordeaux man later slipped into a coma after being poleaxed by goalkeeper Harald Schumacher.
Did Schumacher receive a red card? A yellow? Neither, referee Charles Corver awarded a goal-kick to Schumacher instead.
The game ended 3-3 and West Germany went through on penalties, with Schumacher in goal.
On a Richter scale of referee gaffes, this one comes in somewhere around 8.5—at least in the minds of English football fans.
It was the quarter-finals of the 1986 World Cup and England were facing Argentina in Mexico City. With the scores locked at 0-0, Diego Maradona scored a goal with his hand in the 51st minute.
Referee Ali Bin Nasser did not see the incident and famously let the goal stand.
"I was waiting for my teammates to embrace me, and no one came. ... I told them, 'Come hug me, or the referee isn't going to allow it,'" said Maradona in 2005, as reported on ESPNFC.
Maradona then went on to score one of the best goals in the tournament's history
When Gianfranco Zola was sent off just minutes after taking to the pitch for Italy at the 1994 World Cup, it was a highly controversial decision.
It was a rather innocuous challenge, however the Azzurri man was adjudged to have made a serious foul on Augustine Eguavoen of Nigeria. He was subsequently sent off by referee Arturo Brizio Carter, who dished out nine yellow cards over the course of the game.
To add insult to injury, this happened on Zola's 28th birthday.
In an exclusive interview with B/R in November 2013, Zola recalled the incident. He said:
I was surprised. Shocked. I went from being a player who had received just a few yellow cards in his career to a red card in a World Cup. It was a devastating moment for me. It was my first game in the World Cup, so I was excited and thankful for the opportunity. Being sent off was a big blow.
We're about a third of our way through this slideshow and if you've started to feel sorry for referees, then the clip above might make you think again.
It was the final home game of the calendar year, when Rangers took on Hibernian at Ibrox on December 30, 1995.
Referee Dougie Smith had dropped his yellow card when Paul Gascoigne ran to give it back. When he caught up with the match official, Gazza (being Gazza) decided to brandish the card.
Obviously in want of a sense of humour, the ref booked the Rangers man on the spot.
It does not matter what else Rivaldo has done on a football pitch, for memories of the midfielder will always be tainted by the injury that he feigned in the 2002 World Cup.
Calling it controversial is putting things mildly.
Turkey's Hakan Unsal kicked the ball against the Brazilian's legs and Rivaldo went down like a ton of bricks.
Unsal was dismissed as a result of the incident in the last minute of Brazil's 2-1 win. Later that week, Rivaldo was fined by FIFA but seemed unrepentant about his actions. As seen on BBC Sport, he said:
I'm calm about the punishment. I am not sorry about anything.
I was both the victim and the person who got fined.
Obviously the ball didn't hit me in the face, but I was still the victim. I did not hit anyone in the face.
It was a case of "the goal that never was" when Tottenham Hotspur were denied a legitimate strike against Manchester United in 2005.
Roy Carroll, who was United's goalkeeper at the time, fumbled with a 45-yard shot from Spurs' Pedro Mendes. He dropped the ball, which clearly bounced over the line, before grabbing it to his chest—hoping that no-one was watching.
Everyone who was watching saw that it was a goal, apart from the people who mattered—referee Mark Clattenburg and his assistants.
The game ended 0-0 and the incident is remembered as one of the worst "ghost goal" decisions in Premier League history.
Graham Poll certainly made an impression when he officiated a crucial Group F tie between Australia and Croatia at the 2006 World Cup.
The English referee had a howler of a game, in which he booked Croatian defender Josip Simunic on three separate occasions before finally sending him off.
Poll also failed to award a penalty to Australia after a blatant handball in the second half.
The Guardian's Sean Ingle described it as "a performance that may go down as one of the most error-strewn in the tournament's history."
It was a challenge that lived long in the memory.
Manchester City and Portsmouth were engaged in the second match of the 2006-07 Premier League season when City's Ben Thatcher made an awful attempt to knock Mendes off the ball.
He didn't stand a chance of getting there, so Thatcher's tactic was brute force, as he smashed his elbow into the Pompey midfielder's face. Mendes was knocked unconscious, appeared to have a fit on the touchline and was taken away in an ambulance.
Thatcher received a yellow card from referee Dermot Gallagher.
At the time, it was a highly controversial call. Later, Thatcher was given a retrospective ban of eight games by the Football Association, as reported by BBC Sport.
The Daily Mail showed a video of the incident, which can be seen here.
If there is one football clip that you watch this week, this should be it.
In a bizarre scenario that unfolded in the Paulista Football Federation Cup in Brazil in 2006, referee Silvia Regina de Oliveira awarded a goal to Santacruzense after a ball boy put the ball into the net.
The ball had gone out of play after a Santacruzense shot went wide, but the ball boy brought it back into play and put it over the line. Cue an apoplectic response from Atletico Sorocaba's players.
The referee said she based her decision on what her assistants told her, "I should have trusted my own vision," she said in a radio interview (h/t to the Daily Mail).
If you cast your mind back to Chelsea's Champions League semi-final tie against Barcelona in May 2009, what do you think of first?
Four penalty appeals that were turned down by referee Tom Henning Ovrebo? Perhaps you remember a visibly fuming rant from Didier Drogba into the television cameras? Or how about the fallout that followed the game, in which Ovrebo received death threats and hate mail?
As far as contentious refereeing performances go, this was right up there with the worst of them.
It's fair to say that, to this day, Thierry Henry probably doesn't have many fans in Ireland.
The Frenchman's double handball in a 2009 World Cup qualifier allowed William Gallas to score France's winning goal.
The 1-1 scoreline ensured that the Republic of Ireland would not be in attendance at the following year's tournament in South Africa.
As reported in The Guardian at the time:
Television footage of Wednesday's match, eight minutes into extra time, shows that Henry twice handled the ball in the box to keep it in play, before a short tap allowed William Gallas to score. Though Ireland are widely held to have played better, the goal put France 2-1 ahead on aggregate, sealing qualification for the French—and a national tragedy for the Irish.
When Duisburg took on Frankfurt in the Bundesliga 2 in 2009-10, they really didn't need any help from the referee.
With a comfortable 4-0 lead, Duisburg's Christian Tiffert hit the Frankfurt crossbar with a long-range effort, the ball clearly bouncing outside the goal line.
Inexplicably, referee Marco Fritz let the goal stand and Duisburg won 5-0.
In June 2010, England were taking on Germany in the second round of the World Cup.
Trailing 2-1, a shot from Frank Lampard during a period of dominance for England was not given, despite clearly crossing the line.
It was not even a close call.
Germany went on to win the game 4-1 and England were knocked out of the tournament. Who knows what might have been, had the goal stood.
How Nigel de Jong got away with this challenge is anyone's guess.
In the 2010 World Cup Final, which was being officiated by English referee Howard Webb, Netherlands midfielder De Jong tackled Spain's Xabi Alonso with a chest high Kung-Fu kick.
"It was one of the worst tackles I have ever suffered," said Alonso to BBC Sport.
De Jong received a yellow for his flying lunge.
This is one of the strangest red card decisions you're ever likely to see.
When Ashley Vickers—player/manager for non-league side Dorchester Town—rugby tackled a streaker to the ground, he was sent off with no hesitation by the referee.
Following the 2011 Blue Square Bet South game at Havant and Waterlooville, Vickers told The Daily Mail:
I'm dumbfounded and speechless. A guy ran on to the pitch without any of the stewards getting near him and I thought I was doing them a favour.
My only thought was to get hold of him so we could get on with the game. I managed to grab him and bring him to the ground and the funny thing was the stewards actually thanked me for it.
But the ref decided to send me off and it beggars belief. Their players told the ref not to send me off and their chairman even offered to take a player off to even things up.
No one in the ground, players and supporters alike, will have seen anything like that before and no one will see anything like it again.
In the 2011 Women's World Cup, referee Gyoengyi Gaal allowed Equatorial Guinea defender Bruna Amarante da Silva to get away with one of the most blatant handballs you'll ever see.
In the match against Australia, Bruna caught the ball as it rebounded off the post and held onto it as she took a couple of steps, before dropping the ball and giving it to her goalkeeper.
The referee waved play on.
Australia went on to win the game 3-2 and Gaal later apologized for missing the incident, as seen on the BBC Sport website.
Lokomotiv Moscow midfielder Roman Shishkin was shown a red card for a challenge he made on Zenit St Petersburg's Hulk in the Russian Premier League in April.
One problem—Shishkin did not make the challenge, teammate Vedran Corluka was the guilty party.
Nevertheless, Shishkin was sent off and his side lost 1-0.
The most recent gaffe in our selection, Stefan Kiessling's phantom goal for Bayer Leverkusen never should have stood.
The 29-year-old's headed effort against Hoffenheim went wide, only to slip through the side-netting and land in the back of the net.
With his head in his hands, Kiessling rued his missed chance, only for the goal to be awarded by referee Felix Brych.
A controversial decision—but who can blame Brych for awarding one of the strangest goals you'll ever see?
As reported by John Drayton in the Daily Mail, Leverkusen director of sport Rudi Voller said afterwards:
Of course this is uncomfortable for us because this is not the way we want to win but we can't do anything about it.
Stefan Kiessling was not sure about it either, he didn't have a good feeling either.
(Hoffenheim president) Dietmar Hopp has spent so much money on such a beautiful stadium, you'd think he would be able to spend a few more euros for a decent net.