I'm pretty sure IRL friends have no idea what weeb is, and online I'm usually the one making weeb jokes. Not sure if I've ever been called weeb, but I guess i'd just roll with the joke. Getting offended by being called weeb just makes it more evident that in fact, you are weeb.

Im in my 50s with a wife and 2 kids, I've been studying martial arts for a looong time. I'd probably laugh and say, "who the hell are you? and would you like to fight? (at my age and status, I'm also slightly crazy)"