Alan Schwartz, the last CEO of Bear Sterns, told CNBC's "Closing Bell" that Greenberg influenced many people's lives, including his own.

"[He] had a tremendous impact on my career and my life. I've often said that I learned more watching Ace than I learned from anything else in the business. There will never be another one like him," he said.

CNBC contributor Larry Kudlow also had fond memories of the late banker.

"Having worked for Ace Greenberg for many years, I can tell you he was a canny trader, a great businessman, and a shrewd judge of people," he wrote in an email. "He was also an incredibly generous philanthropist."

"It's hard to imagine a financial services industry without Ace. In many respects, he epitomized the American dream, rising from a clerk to the corner office during his 65 years with the firm," he wrote.

To some, Greenberg was no less than the epitome of Wall Street.

"From scrutinizing the bottom line, to building his business from the bottom up, he was a titan of Wall Street in the truest sense," Ron Insana wrote.