Pointless is a daytime game show, which means its contemporaries are Loose Women and Dickinson's Real Deal and the weird Channel 4 cookery show that's basically the world's most irksome Sainsbury's commercial. And yet it somehow seems wrong to lump Pointless in with that lot. It's become so much more than that.

This might say more about the meandering gaggle of slackers and no-hopers I follow more than anything else but, like clockwork every weekday afternoon, my Twitter timeline will briefly glow with people congratulating themselves on their ability to beat the Pointless contestants. This sort of thing just doesn't happen with other daytime game shows. People don't publicly play along with the Countdown conundrum, or attempt to outfox the woman from The Chase, or try to remember the name of that rubbishy quiz thing that Phil Spencer has inexplicably started to present.

But, four years in, Pointless has developed a following that borders on the slavish. Think how big Wheel of Fortune was back when it was presented by Nicky Campbell, or how omnipresent Weakest Link was in its turn-of-the-century pomp. That's where Pointless is now, and the reason is simple – it's very good indeed.

If you've never seen Pointless – perhaps because you have a job or you hate fun – it's a bit like Family Fortunes, where the contestants have to answer questions that have been crowdsourced. But a couple of factors set Pointless apart from Family Fortunes. First, it rewards obscure knowledge; the aim of the game is to guess answers that none of the public have thought of. Second, and most importantly, nobody involved is a shrieking idiot.

For a few years, Alexander Armstrong seemed prepared to put his name to any old cack. He presented Epic Win, the show that You Bet would be if it had spent its childhood eating paste in a cupboard. He presented The Great British Weather, which essentially consisted of him standing next to the cheruby bloke who won Strictly that time and pointing at a cloud for an hour. He presented Don't Call Me Stupid, which apparently existed.

He's still prone to outbreaks of ill-advised ubiquity from time to time – later this year he'll present something called Prize Island, which sounds like a nightmarish nonstop Day-Glo car crash of the very worst kind – but in Pointless he seems to have found his natural home.

Armstrong gets to relax and stretch out a little on Pointless. He can still be funny, but the pressure's off. If he wants to be charming, he can. If he wants to casually remind the contestants how much cleverer he is than them – which he frequently does – he can. If he wants to explore the minutiae of an answer with his Vordermanesque cleverclogs sidekick Richard Osman, he can do that too. The show's fondness for the esoteric, as well as its deliberately lazy pace, gives him plenty of chances to indulge whatever whims he likes.

Thanks to the relatively recent weekend offshoot Pointless Celebrities – which, incidentally, goes some way to nullifying my earlier "nobody on Pointless is a shrieking idiot" point – the show is starting to break out of its daytime straitjacket. It will never be full primetime fare, it's too bookish and leisurely for any of that, but that's for the best. Jazzing it up Weakest Link-style would ruin its charm.

Since 2009, Pointless has found its own audience, and they're prepared to defend the show to the death. And if that means having to put up with people bragging about how nerdy they are on Twitter every afternoon, so be it.