That wasn't enough to save his job.
The Raptors fired Casey after seven seasons Friday, just four days after the Raptors were swept by the Cleveland Cavaliers in the second round, according to ESPN.com's Adrian Wojnarowski. Toronto went 320-238 under Casey and made five postseason appearances, but the team's playoff struggles led to his dismissal.
"After careful consideration, I have decided this is a very difficult but necessary step the franchise must take," Raptors president Masai Ujiri said in a statement, according to USA Today's Jeff Zillgitt. "As a team, we are constantly trying to grow and improve in order to get to the next level."
After earning a conference finals berth in 2016 and coming within two games of reaching the first Finals in franchise history, the Raptors fell apart each of the last two seasons—thanks in large part to one LeBron Raymone James.
The Cavaliers swept the Raptors out of the second round in 2017 and 2018, the latter being the death knell to Casey's tenure. Cleveland entered the series having just barely scraped by the Indiana Pacers in seven games, only to beat Toronto twice on their home floor before closing things out in Cleveland. The Cavs had no real set rotation and were still juggling around lineups due to their revamped roster, but it mattered not in the sweep.
The Raptors' season ended in multifaceted embarrassment. Not only did they get blown off the floor in a 128-93 loss, but DeMar DeRozan was ejected after a dirty foul on Cleveland guard Jordan Clarkson.
"The last three years have been rough for us, competing against this team," DeRozan told reporters. "Maybe they just got our number, things just don't go right for us, whatever it is, it could be a lot of things. All I know is the last three years they have been the reason why we haven't advanced."
With the Raptors capped out, they really had only a couple options. They could explore the trade values of DeRozan or Kyle Lowry, but it's unlikely they'd find a package that would improve their roster enough to compete for a title.
Moving on from the coach is the easiest deck-shuffling move they can make without tearing the team to its core. Casey will rightfully be billed as the unfair fall guy, but every NBA coach knows that comes as part of the job.