There is an image of a stage setlist from 1994 on Rob Jovanovic's website (he wrote one of the published biographies of Beck). Among the other regular 1994 songs on it ("Blackhole," "Loser," "Puttin It Down," etc.) was a listing for a song called "I Just Wanna Die."

After some initial confusion about what it was (Beck had no idea), it turns out that this was a cover of the Pussy Galore song from their 1986 album, Groovy Hate Fuck. This was Jon Spencer's early band, before he started the Blues Explosion. They're noisy, punky, raw, and Beck has mentioned in the past about how much he loved them. He has toured and recorded with Jon Spencer too.

Anyway, here's a quote (Buzzy no 14, October 1994):

Beck Yeah! Wow. We did Pussy Galore songs, and he would come out and start singin' em. We wouldn't even ask him.