Gordon Brown is flawed and prone to flying into rages, but he is not the monster of the weekend's caricatures, says Mary Riddell.

Politics is a contact sport. Foreign forums, from the Russian Duma to the Alabama state senate, have frequently been enlivened by punch-ups. In Taiwan, MPs used to stage fake brawls in order to curry favour with an electorate which liked its politicians to be tough. Back home, the former Cabinet minister Richard Crossman once flattened a colleague in a Commons corridor. More recently, John Prescott acquired great prestige by thumping a man who had thrown an egg at him. In this company, Gordon Brown is the odd man out. The Prime Minister has never hit anyone.

His denials of physical violence have, however, done little to diminish the notion that No 10 is a modern Babylon, in which Mr Brown plays Nebuchadnezzar to his cowering staff. This parody has been built on the twin pillars of a new book by the journalist Andrew Rawnsley, and by revelations supplied by the National Bullying Helpline.

While ministers and No 10 staffers accuse Mr Rawnsley of being wrong on some (or many) details, few dispute his broader picture of tension at the heart of government. As one of the PM's closest ministerial colleagues says: "Of course there is some stuff I recognise." The Rawnsley account has thus far largely centred on the poisonous climate at the nadir of Mr Brown's tenure when, as another senior minister tells me, "Gordon was helpless with fury at himself for being unable to pull things together."

The second barb to Mr Brown comes from Christine Pratt, whose bullying helpline has, she alleges, been contacted by an unspecified number of No 10 employees anxious about their workplace culture. Disobliging questions are being raised, not least about alleged links between Ms Pratt's charity and a commercial agency run by her husband.

Any distressed Downing Street aide might have noted the entry on the Charity Commission website, recording that the helpline's 2008 accounts are some 200 days overdue. More relevantly, the charity has been challenged about its assertion that: "We have a clear privacy policy. We do not share your details with anyone."

The disclosure has been criticised by Ann Widdecombe, one of the charity's patrons who have now resigned. At this rate, the affair may prove a greater embarrassment to David Cameron, author of a glowing online testament to the helpline, than to Mr Brown.

But the bullying furore, though overblown, is not negligible. It's too simplistic (though true) to say that, in the absence of any killer detail or any challenge to his leadership, Mr Brown can tough this one out, with polls putting him only six points behind a flailing Mr Cameron, and even Lord Heseltine warning that the Tories are heading for a hung parliament.

Equally, it is too facile to paint a caricature of an omnipotent tyrant whose habit of thumping the car seat strikes terror into the shrinking violets of his protection corps. While bullying is never to be condoned, the examples on offer are more likely to brand Mr Brown as Violet Elizabeth Bott than Emperor Nero.

No one has any doubt that the PM possesses the sort of temper that might warrant his temporary exclusion from a nursery class. It is perfectly possible to imagine him flying into such a rage at the prospect of seeing European ambassadors that he yelled: "Why are you making me meet these f------ people?" I'm sure he did use those words, or something similar. The question is why the recipients of such outbursts are, in the main, so unresentful.

Some would surmise they are punch-drunk. In fact, although a few senior Cabinet ministers have grown to dislike or even despise Mr Brown, most staff and colleagues either genuinely admire him or at least consider him worth the pain. He is not only, they say, "sharp, intelligent, impressive", he is also someone with "generosity of spirit". As a veteran of despair in all its forms, he can empathise with the distress of others. "He can be very kind, and very sweet," one colleague says.

Such endorsements are, however, unlikely to cut much ice among those voters to whom the vicious ways of Westminster seem ever more bizarre. An electorate schooled in public tolerance is entitled to be aghast if those it elected, and their retinues, make the Borgias look blander than the cast of Hollyoaks.

While many of Mr Rawnsley's examples of strychnine politics are historical, the culture of The Thick of It remains embedded. Maggie Darling may have less cause, these days, to castigate the enemies briefing against her husband, but old rifts are hard to heal. On the other hand, the Cabinet Secretary, Sir Gus O'Donnell, has belatedly made it clear that he never raised concerns with Mr Brown about staff treatment, so undermining the juiciest story in the Rawnsley account and the demands by Mr Cameron and Nick Clegg for an inquiry.

Despite some legitimate concerns, the current furore is a sideshow. Just as it is no secret that Mr Brown has an explosive temper, it is obvious that he is the prime victim of his tantrums. Despite frequent (and often trivial) eruptions, the bulk of his rage has been directed inwards, at his own inabilities to correct the nation's problems and tap into its heart.

Mr Brown bears scant resemblance to a textbook bully. There is nothing premeditated, or deliberately unkind, in the bursts of frustration that can sometimes be assuaged by Lord Mandelson or braver aides. But who dares tell Mr Brown that his anger is a sign not of strength but of weakness?

I would guess that, were the PM to read Mr Rawnsley's book, he would be less ashamed of his spats than of the feeble decisions dictated, in part, by his temperament. For example, his panicked decision to try to match the Tories' inheritance tax wheeze, rather than calling their bluff and holding an election, now looks like folly. One wishes, too, that the spleen allegedly vented on underlings could have been directed at our over-mighty security services, who now stand accused of complicity not in bullying but in torture.

Mr Brown, as well as being occasionally too rough, is also guilty of the greater sin of being too timid. Despite these flaws, he has pledged to put "character" at the heart of his election campaign. He should be wary: character in a politician is a relative virtue. Tony Blair, though delightful to typists, had few qualms about subjecting the Iraqi populace to the hairdryer treatment and worse.

Voters have a right to know that their leaders are not charlatans or crooks. Beyond that, Mr Brown's combination of rectitude and crossness might be his own business, but for the laser beam of the modern media and the pre-election debates that will shine the strongest spotlight ever on the antagonists.

Mr Brown can win on policy. He can also, if he chooses, give Labour voters what they want, whereas Mr Cameron cannot, on issues ranging from Europe to the economy, satisfy his party's disparate demands. When it comes to character, Mr Cameron is too flexible in his credo, while Mr Brown is too unbending.

Ironically, he had just started to relax, reassured that his soul-baring interview with Piers Morgan had at last struck a chord with voters. Mr Brown, far from being a monster, displays extremes of kindness and intemperance. It is a tribute to his doggedness, his willpower, his resilience and his variable charm that he is still in this election with a chance.

Mr Rawnsley's book was predicated on Mr Brown's impending nemesis – a scenario that the PM has omitted to deliver. The attacks of this weekend are far from fatal for Labour's Houdini. He could even, if he chooses, turn them to his advantage. As one wiser minister says: "You don't get to be PM by being a modulated human being. But there has to be a candid narrative about who Gordon is – curmudgeonly, slightly off the wall, but still a visionary."

Can Mr Brown confront his limitations? We shall see. His frustration on Sunday that the BBC was leading on the bullying story rather than his fightback suggests that he is still blaming others for his woes. It is beyond time for the most complex politician of his age to take the advice of the Cabinet member who urges him: "Live with who you are."

But as the polls point to the resurrection of Gordon Brown, it is just possible that Britain will also have to live with its mercurial leader for longer than the doubters ever imagined.