Talking about the character of Goat Boy, in an interview on Mancow's Morning Madhouse

Everything I did on that show was me screwing around in the hallways, running around looking for attention. Joe Pesci, I never said 'Oh, I wanna do the Joe Pesci Show.' It was me, sitting around, bored out of my mind, and just walking the hallways going [Joe Pesci voice] 'Where do you get your culios big enough to call me...wait, I'm not gonna answer ya! S'ya worryin' about your weight, ya fat bastard, otherwise you wouldn't have problems upstairs doin' ya wife!' And this stoner guy, who was an intern, said 'Hey man, you should do that on [Weekend] Update!' He said, 'Well, "Casino's" coming out, and you should go on as Pesci and have Norm [MacDonald] say he's never heard of it.' Originally, it was me and the stoner intern, and I'd go on and Norm MacDonald's [Norm MacDonald voice] 'Hey, here to see, tell you how to do a movie, is Joe Pesci over here!' And I'd come on, [Joe Pesci voice] 'Hey, blah blah blah blah!' And he goes 'Well, I've never heard of it,' and I knock the crap out of him. And then some writer was like 'Wait wait wait, what are you doing? Hey, I've got an idea for it.' And then we went in...

Explaining the origin of the Joe Pesci skits in an interview on Mancow's Morning Madhouse

The goat thing, I went out, I was drinking some beers with the writers - writers I couldn't stand, and they didn't like me either. They were all like Harvard, Yale, and [imperious voice] 'We've been studying comedy for seven years, and...we've never been on stage, but we know comedy! Bwahahaha!' So I said 'Listen, I know this is a little out there, but what about a guy...who has Tourette's of a goat?' And these guys just stare at me like 'Man, Breuer's HIGH out of his MIND!' I said 'The more he drinks, the more he starts eating the curtains, and he gets nuts and sings karaoke at the end of the night.' And then about two weeks later, this guy came back, he's like, 'Hey, I've got an idea for that weird goat thing you were talking about.' He said 'What if he only sings 80's?' And I thought that was the DUMBEST...thing I've ever heard in my life! And then, we tried it, and now I've got people drunk out of their minds in a bar trying to 'baa' at me. [drunken voice] 'Hey, man! Yo, yo man, it's the sheep dude right here! [drunken 'baa'ing] I shouldn'ta had that hot dog, man!' Just hammered, baaing at me in the street.