Stern assured us he doesn't want the job, and that whoever replaces Letterman had better be good.

"Anybody who takes that job better have something new and interesting to say," remarked Stern.

TV legend Larry King thinks Chelsea Handler could handle the job, saying, "She would be a top choice. She has a good audience on her cable show. I bet they would be considering her."

The timing couldn't be better for the edgy blonde. She announced earlier this week that she was leaving her E! Network talk show.

On INSIDE EDITION's Facebook page, the top vote-getter for the job is Ellen DeGeneres, followed closely by Craig Ferguson and Chris Rock.

TV Guide Editor Steve Battaglio told INSIDE EDITION, "I think CBS will look for someone who can be a big, broad entertainer, who can do a lot of things. Looking in that realm, I think you have Stephen Colbert from Comedy Central. Also, Neil Patrick Harris, who's done very well as a live awards show host."

And get this, Los Angeles Mayor Eric Garcetti, who's still smarting over The Tonight Show's move to New York, is making a pitch to bring The Late Show to the West Coast. In a letter to CBS chief Les Moonves, he wrote that he was: "Excited for the opportunity to encourage you to bring CBS' next late night show to our city, the entertainment capital of the world."

David Letterman told the world he wanted to spend more time with his family, after spending—as he put it—more than half his life in makeup. But could there be another reason for his decision to call it quits?

Battaglio said, "Dave is going to be 67 years old. And I think he sees that a new generation has come on in late night. Jimmy Fallon is doing very well on The Tonight show. It's probably not going to happen that The Late Show is going to catch him."

Letterman told his audience, "Some time in the not so distant future, 2015 for the love of God, in fact, Paul and I will be wrapping things up and taking a hike!"

The shocked Late Show audience gave Letterman a sustained standing ovation when he broke the news Thursday night.

Letterman said the first person he told was Les Moonves. "I phoned him just before the program and I said, 'Leslie, it's been great. You've been great. The network has been great, but I'm retiring.' "

Paul Shaffer said, "Really? You actually did this?"

"Yes, I did," confirmed Letterman.

True to form, Letterman played the dramatic news for laughs with bandleader Paul Shaffer, joking, "What this means now is that Paul and I can be married."

Even President Obama reacted, tweeting: "There are more than ten reasons David Letterman will be missed."

Funnyman Howie Mandel can't believe Letterman is leaving, saying, "The thought that he won't be on is, at this moment, unfathomable!"