However, Letterman didn't let go that easily. Although Chappelle said "I don't talk about" leaving the show, he did get at least semi-serious when asked whether he regretted turning down all that money. "Of course I would like to have that money," Chappelle, 40, said. But while he admitted "money gives me choices" when comes to things like where to send his kids to school, he also stressed that "it's not the end-all, be-all. There are other things in my life that I do not purchase with money that are very valuable," he said.

Watch Chappelle's sit-down with Letterman below, in which he also talks about his infamous trip to South Africa: