Cavaliers general manager Koby Altman is not among those who share that opinion.
Cleveland is coming off four consecutive NBA Finals appearances, but James' supporting cast received a lot of criticism during this last postseason run. In fact, Saturday Night Live even created a sketch to mock "The Other Cavaliers."
And yet, Altman has no interest in starting from scratch, which would mean trading All-Star Kevin Love and others.
In his first public comments since James' decision, Altman made it clear to Cleveland.com's Joe Vardon that the franchise will ride things out:
"The years where you're non-competitive aren't fun, and I know that's hard to build a culture that way and we've had a great culture of winning because of LeBron and that infrastructure that he brings right away. And I'd like to capitalize on that.
"You won't be nearly as good if you lose a player of that magnitude. But we can still be competitive, we can be tough, we can be skilled, we can be talented and we can still use that culture that's sort of been embedded these last four years.
"To go the complete opposite direction, which we've been through, is not fun and it's hard. It's hard to sort of come out of that. And the unknown is really daunting to me. I really like what we have now and I don't necessarily want to go backwards."
When asked about the possibility of trading Love, Altman told Vardon that he views the forward as part of the team's future:
"Kevin is an All-Star and you don't get better by moving Kevin. Kevin's been incredible for us for four years and he wants to be here, and to me that's a big part for guys that are here and the guys that we're gonna acquire, is that they want to be here and be a part of this new chapter and culture that we're creating."
That echoes what ESPN.com's Dave McMenamin reported back in June, noting the organization wanted to hold on to Love regardless of James' decision.
You don't have to tell Cleveland how hard life can be without James. When the four-time NBA MVP left the first time around back in 2010, the Cavs averaged just 24 wins per season over the next four years. They wound up picking No. 1 overall three times during that span, although one of those picks came via the Los Angeles Clippers.
With Love, rookie Collin Sexton and Co., Cleveland likely won't bottom out the way it did after "The Decision," when it went 19-63 in 2010-11. The Cavaliers will likely compete for a playoff spot in the Eastern Conference next season, but it would be stunning if they found their way back to the Finals.
The Cavs acquired Rodney Hood, Jordan Clarkson, Larry Nance Jr. and ultimately Sexton to try to protect themselves in the event James decided to leave again. Now, they will be able to find out just what they have.