Veteran broadcaster David Dimbleby has hit back at claims by comic Dara O Briain that Question Time needs to tackle the gender balance of its panellists, saying he had picked "the wrong target".

He defended the show's record for inviting women guests and pointed out the proportions were much higher than those for many areas of public life.

Dimbleby's comments come after O Briain turned on BBC1's political discussion show in a Radio Times interview after being asked about a recent decision to ban all-male line-ups for comedy panel shows.

In the interview he observed: "It's remarkable that this amount of time is spent debating women on comedy shows rather than, say, Question Time."

But stung by the suggestion that his show had done anything wrong, Dimbleby said: " Seeking to deflect attention from the gender imbalance on comedy panel shows Dara O Briain suggests that critics look instead to Question Time. He has picked the wrong target.

"Of the 104 panellists in the present run 46 have been women (44%). Given that we invite guests prominent in public life this is not a bad record. Only 18% of the cabinet is female. Only 23% of MPs are female. Only 20% of FTSE board members are female. Our aim is to achieve a 50/50 balance but British public life has to catch up before we can.

"Mr O Briain - the mote is in our eye, the beam is in yours."

In the interview O Briain had expressed his view that scrapping all-male comedy panels would mean viewers would see any female guest as a "token woman".

It stemmed from BBC director of television Danny Cohen pledging that "we're not going to have any more panel shows with no women on them. It's not acceptable."

But O Briain suggested viewers will now view guests in a different light and said if changes were needed they should have "evolved".

"I wouldn't have announced it, is what I'd say. Because it means Katherine Ryan or Holly Walsh, who've been on millions of times, will suddenly look like the token woman," said the maths and theoretical physics graduate, soon to host a new series of School Of Hard Sums for the Dave channel.

"It would have been better if it had evolved without showing your workings, if you know what I mean. Legislating for a token woman isn't much help.

"A certain number of women want to go into comedy, and they should be cherished and nurtured, but you're not going to shift the fact that loads more men want to do it."

Defending the efforts to impose more of a gender balance, a BBC spokesman said: "We're confident that a wider range of voices and opinions will make for even better telly."

O Briain later sought to clarify his comments, saying he was uncomfortable about the way the ban had suddenly been announced, rather than more subtly introduced.

In a message on Twitter, he said: "I have no problem with a policy of no all-male panel shows. I just wouldn't have announced it.

"The same very funny women will be on all these shows and don't deserve anyone to regard that booking as being because of an edict."