We do not need to linger too long over the spray of venom at the end of that sentence except to observe that it might teach the prime minister the perils of seeking a cheap laugh in the Commons at the expense of one of your more eccentric backbenchers. When he humiliated Ms Dorries at prime minister's questions some months ago, he forgot to beware the fury of a woman scorned. It is the label "arrogant posh boys", a phrase Ms Dorries likes so much that she uses it twice to damn her leaders, that threatens to stick. This could not be so, of course, if she was alone in her opinion.

In many ways she has a deserved reputation for being a one-off, but on this occasion she distills what is close to now becoming the orthodoxy within their own party about the two most senior men in government. One reason her tirade resonates is because she is saying in public what many others Tory MPs also say and have said for some time, just a little more discreetly. The charge sheet is this. The government is led by a clique of toffs who have neither respect for their colleagues, nor empathy with the average voter. Their born-to-rule mentality means they have a greatly over-inflated view of their own capabilities, which deafens their ears to the advice and warnings of others who might actually know better. They are nothing like as good at governing as they think they are. And this, the charge sheet concludes, is now inflicting serious harm on both the country and the Conservatives' future electoral prospects. This view is now becoming more and more prevalent in the media, too, even among the press that the Conservatives would normally count as their friends.

It is the "boys" that is the real killer. You can get away with being regarded as "arrogant" at the top of politics. The careers of Margaret Thatcher and Tony Blair both demonstrate that being seen as domineering, and its close cousin, "out of touch", does not have to be a barrier to being re-elected. In fact, voters are bound to assume that politicians are intrinsically arrogant people. By its very nature, politics is not a trade that tends to attract the humble. To believe that you are better qualified to run the country than anyone else is, by definition, arrogant. The public may even mistrust politicians who do not have some aura of arrogance about them because it implies a lack of confidence in what they are doing. Ed Miliband would benefit from a touch more hauteur if he is to grow into a convincing candidate for prime minister.

Once asked, while in opposition, why he wanted to become prime minister, David Cameron replied: "Because I think I would be quite good at it", one of the most self-revealing remarks he has ever made. Shortly after he moved into Number 10, someone inquired whether anything about the job had come as a surprise to him. Not really, he insouciantly replied: "It is much as I expected." In his early period in office, that self-confidence served him rather well. He certainly looked quite good at being prime minister. He seemed to fit the part and fill the role. Broadly speaking, he performed like a man who knew what he was doing. That is one reason why his personal ratings were strikingly positive for a man presiding over grinding austerity and an unprecedented programme of cuts.

So you can get away with being "arrogant" as long as the voters think you have something to be arrogant about. You can also get away with being "posh" in politics. To most of the public, anyone who wears a good suit and swanks about in government limousines looks "posh" whether their schooldays were spent at Eton or Bash Street Comprehensive. There may even be some voters who think – or at least once thought – that an expensive education has its advantages as a preparation for running the country. Though David Cameron and George Osborne have always been sensitive on the subject, poshness wasn't a really serious problem for them so long as they could persuade the public that they were in politics to serve the interests of the whole country, not just of their own class. That was why they took such a risk when they cut the top rate of tax while austerity was still biting hard on so many less affluent voters.

At a recent meeting of cabinet ministers and their advisers, most of them managed to persuade themselves that this would not do long-term reputational damage to the government so long as the public could be convinced that they did it to encourage entrepreneurship and boost growth for the benefit of all, not because they wanted to give a big bung to the Tory party's rich friends. It is in the respect of perceived motivation that the revelations from the Leveson inquiry intersect with toxic effect on the fall-out from the budget. Most of the public may not be following every seedy twist and sleazy turn of the Conservatives' relationship with the Murdoch empire with the same fascination as politicians and journalists – the government is rather relying on the hope that they aren't. But if all the public takes away from it is that ministers bent over double to lubricate the commercial ambitions of a hugely rich man – hoping, in the process, to serve their own electoral interests too – that is quite damaging enough. As with the earlier "cash for Cameron" scandal, but on a much more serious level, it makes it even less likely that the public will invest trust in the values and intentions of ministers, which will bleed across into public attitudes towards everything else they say and do. At its lowest, it makes them look like men who put party interest before national interest. It feeds a picture of a government of the rich by the rich for the rich.

Even then, they might get away with it, especially if most people were feeling fairly prosperous themselves. The money scandals that regularly erupted around Tony Blair did not prevent him from winning three elections in a row. However loudly the headlines screamed at the time, hostile reaction among voters was softened by a booming economy.

That insulation from public discontent is not available to a government ruling at a time of painful austerity, which makes it much more vulnerable to explosions of scandal. The vivid theatre of the Leveson inquiry is made more dangerous for the government because of the announcement from the drier forum of the Office for National Statistics that the economy shrank in both the final quarter of last year and the first quarter of this year, meeting the technical definition of a "double dip".

Now, it maybe that these figures will be subsequently revised upwards. It may even be that the chancellor's friends are right, if a bit desperate, when they say that "most voters probably thought we were in recession anyway". But it is inescapably true that the return to growth that they had promised the country and themselves by this stage has not materialised. Either their economic strategy was wrong from the start or they under-estimated the scale of the challenge facing them when they took office.

Since they can't admit to such a fundamental error of judgment, they will have to confess instead to being naive about what they took on.

By historical standards, David Cameron and George Osborne were both boys when they got the jobs. Mr Osborne was the youngest man to become chancellor since Randolph Churchill in 1886. Mr Cameron was the youngest man to become prime minister since the Earl of Liverpool at the beginning of the 19th century. Neither man had held any ministerial position whatsoever before they took on the most powerful roles in British government.

In their early phase in office, they did rather well at inducing most people to neglect that basic fact about their biographies. When they made mistakes and they made plenty in their first months in power, they tended to be ignored or forgiven because of the general goodwill for the coalition in the media and among the voters. Now, many of their earlier errors are coming back to bite them and at the same time there is a pile-up of contemporary blunders, schoolboyish in their combination of amateurism and slapdashery.

Unless something changes, and pretty quickly, "arrogant posh boys" will be glued to Messrs Cameron and Osborne as indelibly as "something of the night" was to Michael Howard.

• This article was amended on 3 May 2012. The original said that Randolph Churchill became chancellor in the early years of the 20th century. This has been corrected.