Kovalev, who laughed through much of the exchange, noted that he previously predicted he would end Ward's career before pointing out that Ward retired after the two men had fought.
The two men fought two times in the past year, with Ward winning in November by unanimous decision and in June by TKO. Kovalev believed he got robbed in the first fight, however, and should have been given the decision.
Finally, Kovalev believes Ward's retirement will motivate more boxers to seek the light heavyweight title and boxing fans won't be subjected to what he called "more boring fights."
"Of course I'm disappointed that I don't have the opportunity to kick his ass again," he said in the interview, before emphasizing "again" for a second time.
Ward, on the other hand, didn't mention Kovalev in his decision to retire.
"I want to be clear—I am leaving because my body can no longer put up with the rigors of the sport and therefore my desire to fight is no longer there," he said in a statement, per ESPN.com. "If I cannot give my family, my team, and the fans everything that I have, then I should no longer be fighting."
He also spoke during First Take about the rigors of the sport.
"People see what I do fight night. They see under the lights, but they don't see the toil, they don't see the grind, they don't see just the pain, the physical pain that you go through, not just in the fights, but to prepare and to get ready for those battles."
Ward continued: "I felt the physicality of the sport—not just in-the-ring stuff, but the training and the preparation—start to take its toll on me for the last two or three years, and I bit down and continued to push through. And at this point, it's time, and I know it's time."
As for Kovalev, he's scheduled to face Vyacheslav Shabranskyy on Nov. 25 at Madison Square Garden.