Behind Matt Damon's Raunchy Payback to Jimmy Kimmel

PEOPLE's Sexiest Man Alive became an instant YouTube sensation on Friday following his duet with comedian Sarah Silverman, as she revealed to her boyfriend, late-night host Jimmy Kimmel: "I'm f---ing Matt Damon."

The line was an instant classic. But for many Kimmel fans, the video was just the latest salvo in the long-running "feud" between Damon and Kimmel. It all started during the third season of ABC's Jimmy Kimmel Live, when the host made a tradition of closing out episodes with, "Apologies to Matt Damon, but we ran out of time," even though the actor was never scheduled to come on the show.

The joke continued in September 2006 when Damon finally appeared as a guest. (See the clip.) After a segment which explained the gag – and a lengthy introduction by Kimmel – Damon sat down, only to have the host cut him off and end the show. As planned, the actor unleashed a flurry of curse words on Kimmel and proceeded to storm off the set. His performance was so believable, many fans thought he was truly upset.

The gag continued into 2007, employing Kimmel's sidekick, Guillermo – once while interviewing Damon on the red carpet at the Oceans 13 premiere, and again when the show premiered the "unofficial" trailer for Damon's film, The Bourne Ultimatum.

'His Turn to Get Jimmy'

Producers at Kimmel approached Damon this past October to appear on the show for the host's 40th birthday celebration but he opted to keep the joke running.

As for creating the video, "I don't even think he thought twice about it," his rep tells PEOPLE. "He just said, 'I'll do it.'"

Damon shot the video with Silverman at the end of October in Miami. "He had so much fun," his rep adds. "It was his turn to get Jimmy."