If you thought that Stephen Colbert’s live election night special on Showtime was one of the most amazingly awful pieces of television you’ve ever seen, the Late Show host would like you to know that it was doubly awful being in front of the camera. “That was the hardest thing I’ve ever done,” Colbert confessed during a one-on-one conversation with fellow comedian/political commentator/all-around funny guy, Last Week Tonight‘s John Oliver, held on Nov. 19 at the New Jersey Performing Arts Center as a fundraiser for the Montclair Film Festival. “The audience was sobbing openly.”

What made the telecast even more challenging was the fact that Colbert and his writing staff had to jettison much of their preplanned material when it became clear which way the electoral winds were blowing. “We had guests and pretaped pieces for one of three eventualities: Hillary Clinton wins and we know; Hillary Clinton wins and we don’t know, because it’s not called until the show’s over; or Donald Trump wins and we don’t know, because he has such a narrow path to victory.” In the end, Colbert wound up having to put on a show that nobody had prepared for: Donald Trump winning the election in real time. “We have two-and-a-half shows worth of material that you’ll probably never see,” he said mournfully. “We had all these made-up commercials, but none of them were appropriate once we went on the air.”

Two weeks removed from that difficult election night, Oliver and Colbert enjoyed a free-flowing conversation covering everything from their Daily Show days to what to expect from the next four years. Here are five other highlights from this meeting of two of our favorite comic minds.

Bomb hardEvery standup comic has a “Worst Gig Ever” story, and Oliver was kind enough to share his tale with us. One night, he took the stage to perform for an audience of four, all of whom left before the hour-long set was over. “Two people left about 15 minutes in,” he remembered. “So we’re down to two people, a married couple. And then the husband — in a real act of betrayal — said to his wife, ‘I’m just going to the toilet.’ We knew that wasn’t true, because he took his coat with him. So he leaves, and it’s just me and his wife, and I’m trying not to look into her eyes. I was approaching the 30-minute mark, and I saw her hand go down toward her bag. I said, ‘You’re leaving, aren’t you?’ And she said, ‘I’m sorry.’ I could hear the door as she opened it, and when it closed, the lighting guy said, ‘Do you want to keep going?’ There’s no way to spin that as a good gig!”

Nothing like a pillow fight between friends. (Dave Kotinsky / Getty)

The naked timeBeing on HBO allows a late-night host certain freedoms that a late-night host on, say, CBS wouldn’t get — freedoms like four-letter words, no commercials, and, of course, lots and lots of nudity. And Oliver remembered that Last Week Tonight made sure to take advantage of the latter option, which is readily employed by such popular series as Game of Thrones and Girls, early on. “In the first season, we did one scene of full frontal male nudity — not mine! — and we had to cast a series of penises, so they brought me all these penis shots. My casting criteria was: [the penis] shouldn’t make your heart hurt, or feel bad for the person or make you think that a trip to the doctor would be advised. So that eliminated two out of the five penises.” Now that he’s in his third season, though, he doesn’t feel the need to strip down again. “We wanted to exercise all those muscles [early on] and then relax and do our show. But we couldn’t not do it at that point.”

No guests, please, he’s BritishPop quiz: What do Last Week Tonight and Full Frontal With Samantha Bee have in common beyond the fact that they’re each hosted by ex-Daily Show correspondents? They also both eschew the typical late-night format of making time for an interview portion. In Oliver’s case, that wasn’t his original intention. “We built a whole side of our stage as an interview area, because that’s what I thought you did. That’s the DNA I was raised on at The Daily Show. You talk for a bit, and then you to talk to someone else for a bit, and then everyone stops talking.” The host did incorporate interviews into never-aired test shows, but dropped them after HBO said that he could use that time for other material. “We still have this whole interview stage that we barely ever use. Right now, it’s where we have dancing mascots and explosions.”

All the (fake) news that’s fit to printAs veterans of the Jon Stewart era of The Daily Show, both Colbert and Oliver took umbrage at the postelection complaints that have been circulating about “fake news” — those invented stories from dubious sources that were widely shared on Facebook. “What we did [on The Daily Show] was fake news,” Colbert said. “We got on TV and said, ‘All this stuff is going to be fake; we’re going to make fun of news.’ The fact that they’re calling this stuff ‘fake news’ upsets me, because this is just lying.” And speaking of The Daily Show, both hosts agreed that new host Trevor Noah is doing a fine job in Jon Stewart’s old chair.

The elephant in the roomAfter spending the first half of their conversation talking around the biggest news story of the past two weeks, the hosts steeled themselves to tackle the incoming president head on. At Colbert’s prompting, Oliver elaborated on his “This is not normal,” message from the recent Season 3 finale of Last Week Tonight. “The danger of saying, ‘Just live your lives, the sun still comes out tomorrow’ is that it’s true for some people, and it’s very easy to forget that it’s not true for others. Not everyone is going to be OK, and it’s incumbent upon everyone to remember that. You have to keep remembering that this is an abnormal version of what we’ve been through before.”

Later on, during a lively audience Q&A, talk turned to Trump’s recent Twitter scuffle with the Hamilton cast, and Colbert had some positive words for Vice President-elect Mike Pence. “I like that he stayed and listened. He actually stayed in the doorway and listened, and they thanked him for staying to hear it.” Oliver was less impressed: “He stayed in the doorway. That feels like the Mike Pence version of listening.”

(Photo: Dave Kotinsky / Getty)

Colbert also shared his own feelings about the calls to give the president-elect a chance. “I’m all for ‘Give him a chance,’ but don’t give him an inch. Because I remember everything he said [during the campaign], and it’s horrifying.” But Colbert wasn’t all doom and gloom: He also offered a three-step prescription for how to handle the next four years. “Vote on the local level, get involved in local organization, and get to know your neighbors. There are reasons to get together that have nothing to do with politics. There have to be things that we do that are not competitions, and politics are jockeying for power. So get involved in your local community in significant ways that help you meet your neighbors.”

The Late Show With Stephen Colbert airs weeknights at 11:35 p.m. on CBS. Last Week Tonight with John Oliver will return Sunday, Feb. 12, on HBO.