So Tony Abbott regards Julia Gillard as the worst prime minister since William "Billy" McMahon.

An interesting comparison, not so much for his view of the person occupying the job he so brazenly covets (he would say that, wouldn't he?), but in his rare admission as a Liberal that one of his own had been so bad.

By broad agreement among political historians, McMahon generally comes in at 27 on the list of 27, but was he really all that bad and, if so, how?

Curiously, it was not so much what he did in the 19 months he served as leader of an ageing, out of touch and dying government (he actually did very little), but rather what he was: a liar, a leaker, a plotter – and worse. All those who had worked with him came to despise him, and former colleagues of stature, such as Bob Menzies and Paul Hasluck, referred to him as "that despicable bastard" and "a contemptible little quirt."

Certainly as prime minister, McMahon cut a ludicrous figure – a man captive of his own fantasies and seemingly not of this world. Gravitas was not one of his qualities. Nothing so typified this as an interview with a correspondent for an international news magazine who, as his last question, asked McMahon what his view on the future of Australia was. Rummaging through a weighty briefing document his department had prepared for him, he opened it at F but found no answer. To the astonished correspondent, McMahon said absentmindedly, "No, nothing on the future here."

His predecessor, John Gorton, dumped in 1971, had suffered a massive swing in 1969, wiping out in one stroke the comfortable cushion of Harold Holt's record 1966 landslide, and the most optimistic hope for McMahon was as a caretaker who might manage to hold some seats in 1972 and keep the party within striking distance. McMahon, for all his faults, did just that – but for many Australians, Liberal supporters included, he inspired little confidence and his prominence as the public face of Australia was as embarrassing as it was bizarre.

What made things worse for McMahon was his opponent, Gough Whitlam. Intellectually acute, articulate, suave, policy focused and a brilliant parliamentary performer, he was all the things that McMahon was not, and the real surprise of Labor's win in 1972 was not the win itself but the very narrowness of it.

Whitlam woke up a sleepy Australia, but it was an energetic and forward-looking prime ministership, hampered at times by a ministry of limited ability, and marred at times by questionable judgement. It was he, for example, who appointed his eventual nemesis, John Kerr, to the job of Governor-General in spite of warnings from colleagues.

Malcolm Fraser who followed in the wake of the 1975 crisis presided over a fractious ministry – fractious mostly in response to Fraser's aggressive management style that some might call bullying. Policy wise, Fraser left a slender legacy – the establishment of SBS perhaps his high point – but he deserves kudos for courage in his humane response to the refugee crisis after the Indo-China conflicts and for his principled stand against the minority white regimes in southern Africa and his advocacy of international sanctions (This flew in the face of the Liberal Party at the time, which had strong pockets of support for the apartheid regime in South Africa and the Smith regime in Rhodesia).

Bob Hawke, who defeated Fraser in 1983, was possibly Labor's finest prime minister – a reformer and a conciliator, and his policy initiatives did more than any other prime minister in positioning the Australian economy more advantageously in a rapidly globalising world. But did he stay too long and distract his government too much by resisting an orderly succession?

Paul Keating, who succeeded Hawke after knifing him, will be remembered, I think, as a transitional figure whose deeds never matched the rhetoric. Keating had a vision of a new Australia – a republic, closer engagement with Asia and justice for the Indigenous people – but he never managed to explain or sell this vision in a convincing way, and he refused to see how his structural reforms had created losers as well as winners – and One Nation was born as a result. His finest moment was surely his Redfern apology speech and his strong support for Mabo – but it was also on his watch that the Migration Act was amended to provide for mandatory detention of irregular arrivals.

John Howard defeated Keating in 1996 and, like Hawke, instigated waves of change, most notably to the tax system with the bold move to introduce the GST. Howard's legacy will be long debated, but he was a formidable political operator, way too smart for Labor during the decade of his ascendancy. Howard perhaps read the Australian people better than he is generally given credit for, but the other side of the coin was a sly pandering to populist sentiment – not the hallmark of a great leader.

Then comes prime minister number 26, Kevin Rudd, who returns Labor to office in 2007. Like Keating, he will be remembered for a gracious and eloquent gesture to the Indigenous people with his apology to the Stolen Generations, but his abandonment of the emissions trading scheme commitment after nominating it as the dominant issue of the age, will always call into question both his judgment and his political courage. Much of Australia outside the Canberra hothouse still fails to comprehend the coup against Rudd in mid-2010, but it is doubtful if a more toxic relationship between a prime minister and his senior ministers and officials has ever existed. Rudd simply did not understand his obligations to his primary constituency by virtue of which he held office – his own party. Such recent memories should put to rest all talk of a Rudd return, but stranger things have happened.

Which of course brings us to the present prime minister, and the recipient of Abbott's "worst since" award. Julia Gillard, to be sure, has an image problem and Abbott's mantra of the worst government even appears to gain traction if only for its constant repetition, but arguably for all the wrong reasons.

First, the image. It was never going to be easy selling her as the replacement for Rudd to the electorate, even if it made all sorts of sense. And she never has (but there might be some merit in letting this one go). But scraping back into office at the head of a rather fragile minority government was the game changer – and the volte face on the carbon tax was a pragmatic concession, even though it was going to hurt. The offering up of Harry Jenkins as a sacrifice, the abandonment of Andrew Wilkie in the face of internal revolt over the pokies and, lastly, the outing of union leader Kim Sattler over the Tent Embassy affair to save her own skin, are not the stuff of image enhancement when your trust is being publicly questioned.

Gillard's prime ministership has to be seen in the context of minority government, a situation that none of her immediate predecessors faced. Any knee-jerk judgment of Gillard needs to take this into account if only for the fact that the dynamics of minority government require a precarious juggling act to keep them working. Because she lacks a majority in the House of Representatives, she needs to be more pragmatic and make more compromises than would otherwise be the case. Other leaders have merely had to negotiate with a hostile senate, which pales in comparison (And it will be interesting to see how Tony Abbott as prime minister will deal with the Greens who will inevitably hold the balance of power in the upper house).