Kimmel, who paid tribute to Letterman at the Kennedy Center Honors in December, didn't break the news to Letterman himself.

"I try not to bother him, and I never contact him," Kimmel tells Fresh Air's Terry Gross. "Occasionally, I'll send him a thank-you note or a little note in writing. But I just, I feel like I know Dave pretty well even though I don't know Dave very well, and I get the idea that he's not looking for a lot of interaction. But he did call me and congratulate me and wish me luck on it. He understands how important he is to me, and he's been very nice."

Kimmel fell in love — and that is the phrase he used at the Kennedy Center Honors, "fell in love" — with Letterman as a teenager in the 1980s.

"I didn't have a VCR," Kimmel says, "and so I would have friends tape [Letterman's] show for me — friends who had a VCR — and then I'd go from ... house to house, depending upon who would tolerate it and who would go along with it, and who had the best set of rabbit ears on their TV. And I'd watch the show when I couldn't stay up late enough to watch it. ... It's just one of those things I know we'll be telling our grandkids. ... It's our version of walking without shoes through the snow to school."

In its new time slot, Jimmy Kimmel Live will remain the family affair it has always been — a characteristic that has set it apart from other late-night talk shows. Kimmel's cousin writes for the show, his brother is a director on it, and his best friend from childhood and his best friend's father are in the house band. Kimmel's Uncle Frank — a retired New York City cop and security guard — was a recurring character until he died in 2011.

"The first person that I brought on the air was my Uncle Frank," Kimmel says. "He was living in New York, and the only reason that he moved out to take the job on the show was ... he was worried about transferring his checking account. He had a specific bank at which he had a checking account, and he found out there was one branch of the bank in L.A. So he rented an apartment right near that bank, and that's what made it OK to come out."

That said, Frank didn't initially understand that the plan was for him to be on camera.

"He was very confused," Kimmel remembers. "He thought he was going to be the head of security for the show."

Interview Highlights

On his Uncle Frank's routine after family dinners

"My parents ... didn't have friends, and my aunt and uncle didn't have friends, and my grandparents didn't have friends. They were each others' friends. So pretty much every weekend we'd go to their house and we'd have dinner, or they'd come to our house and have dinner with us. ... We'd have dinner at around 6:30, 7. My Uncle Frank would clear all the plates off the table, and then he would announce that he had to go to bed. So he'd go to his bedroom. He'd say goodnight to everyone, and then, once an hour till about midnight, he'd come out of his bedroom in pajamas and go, 'Chippy, did you pay the water bill?' And she'd go, 'Yeah, Frank.'

Related NPR Stories

"So, he really wasn't going to bed. He was just trying to get away from everyone, and when he cleared the plate, it wasn't because he wanted the house to be neat. He just wanted us all to go home. ... Wherever he was, he was ready to leave, and wherever he was headed, he was ready to go there."

On his favorite prank

"I was driving with my Aunt Chippy, and she was complaining about people who come to this country and don't learn to speak English and it was ridiculous. ... She just goes on this long, xenophobic rant, and [my cousin] Sal and I looked at each other and we realized that this was a weakness and this was ... a vulnerability we could expose, and so I put that in my memory bank.

"And something I've always wanted to do is paint somebody's house while they're away at work. So, while my Aunt Chippy was out at work, we had a bunch of actors pretending to be painters — [and] also pretending that they don't speak any English — paint her house orange and green. And she comes home from work and her house is bright orange and green, and she starts yelling at the workers, demanding to know what's going on. And none of them speak English, which makes her madder and madder, and then there's a guy on the roof who hands her a beer and she goes crazy and throws it at him. They're painting the trees in her yard. It was probably my finest hour."

On those who got angry at him for getting parents to prank their children by telling them they ate all the Halloween candy and then recording the kids' responses

"Kids cry like 40 times a day. ... You know, if I made an adult cry I'd say, 'Oh, my God, this is terrible.' I'd really be upset, but, you know, kids cry constantly and ... there's something very cute about it, and I also think that these kids will treasure that video when they're older and when they look back on it."

On how the running joke about running out of time for Matt Damon on the show began

"We had a bad show. ... The guests were bad, and I was feeling pretty bad about myself at the end of the program. And I decided to say, for the amusement of one of our producers who was standing next to me ... 'I want to apologize to Matt Damon. We ran out of time.' ... And he got a kick out of it, the producer, so I just started doing it every night to amuse him.

"Matt Damon was just the first name that popped into my head. I was trying to think of an A-list star, and somebody we absolutely would not bump if he was on the show. ... The legs on this bit are unbelievable to me. I mean, people laugh every time I say it. ... Repeating the same joke every single night, you'd think eventually people would get tired of it, but they don't."