Federer, considered by many the greatest player the sport has produced, marvelled at Williams' rise to the top alongside sister Venus and father Richard.
Federer then clarified his original opinion by saying Williams may be the greatest "overall," not just among women tennis players.
The distinction made by the word "overall" is a significant one from the Swiss. It means Federer isn't limiting Williams' greatness to her dominance of the WTA circuit.
Instead, he's recognising her skills and accomplishments as distinctive in any arena of the sport. Gay noted the 36-year-old recounted numbers and titles attributed to Williams and Steffi Graf superior to any of the greats in men's tennis.
However, Federer didn't downplay his status in the history of the sport:
"But we know [Serena] is all the way up there. I'm probably up there with somebody, somehow. Maybe there's a group, a best of five—and if you're in that group, you should be pleased and happy. Tennis is a funky sport when it comes to that stuff.
"I'm in full admiration of Serena. And Venus, too, by the way."
Gay also told how he mentioned the comment to Williams, who replied: "I think we can say that about both of us, respectively. He's done amazing things in his career—I have the utmost respect for him."
The records make Williams' case as the best ever hard to argue against. She's won 39 Grand Slams as both a doubles and singles competitor.
A more direct comparison with Federer shows Williams' 23 singles titles eclipse the 20 he has to his credit. Both will be involved at Wimbledon in July when Williams will bid for an eighth title at the All England Club, while Federer will aim for his ninth triumph.