Sugar Ray Robinson being held aloft by other boxers (Courtesy of the New York World-Telegram and Sun via Wikimedia Commons)
Why it should have happened: Charley Burley was one of those fighters who was too good for his own good, and was never able to land a big fight.
He's widely regarded as perhaps the greatest fighter to never fight for a world title, and his legend has grown steadily over the last decade or so.
Robinson, on the other hand, is widely regarded as perhaps the greatest fighter of all time, period.
With their sublime styles, this fight would have been excellent to watch.
Why it never happened: This fight never surfaced for a few reasons. Burley was snake bitten, being too good and too black to make himself an attraction, and was especially ignored by the boxing establishment after beating the white Fritzie Zivic not once, but twice.
Additionally, Burley had been a pro for four years by the time Robinson made his debut, and was retired 15 years before Robinson hung them up.
What would have happened: The timing issues indicated above mean that the prime opportunity for this matchup would have been around 1948, when Robinson began flirting with moving up from welterweight to middleweight.
Burley was on the downside of his career, while Robinson was just gearing up for the great things he would do.
Additionally, Robinson didn't suffer his second loss until 1951. Burley was big enough and good enough to give Robinson trouble along the way, but Robinson would have won by fairly wide scores.