Peter Wilby (the Guardian, 5 September) argues that the manifest failure of neo-liberalism to fulfil its promises to voters now provides the Labour Party with an opportunity and a duty to move on from New Labour’s preoccupation with the centre ground and to assert an alternative and more radical agenda.

He is right of course, but his argument can be taken further. I argue that the whole New Labour experiment was based on an unnecessary defeatism in the face of what seemed to be the established neo-liberal hegemony, and that the consequence was a flawed analysis of what was needed to resist and defeat it.

The acceptance that British voters had endorsed an all-embracing right-wing agenda led the Labour party to conclude that it would have to accommodate this apparently permanent change by making a wholesale shift of its own political agenda along the spectrum to the right.

Accordingly, much of the neo-liberal agenda was adopted by New Labour, sometimes with considerable enthusiasm. The remnants of any vaguely radical policies were quickly jettisoned and new ones eschewed. When the 1997 election was won, the strategy was seen to be vindicated; little account was taken of the fact that victory had been virtually certain in any case because the electorate – fed up after 18 years of the Tories – was determined on change.

The lurch rightwards reflected a common failing on the part of political activists – the assumption that everyone else also holds political views that are internally consistent and together constitute a coherent political stance. But most people are not so considered about politics; they hold views that are often inconsistent and contradictory. They are perfectly capable of nodding assent at any given moment to propositions from every part of the political spectrum.

For many, what determines how they vote is which issues are at the top of their minds as they enter the polling booth. It is here that the right have traditionally gained an advantage; they have been adept at using their superior access to the media to “tweak” those concerns, about, for example, “social security scroungers” or immigrants or increased taxes, that suit their purposes.

New Labour failed to grasp that this simple point meant that moving their whole stance rightwards was both unnecessary and unhelpful. It confused many voters as to what it was they were asked to support. It failed to convince others who found the Tories more credible exponents of the neo-liberal agenda. And it disappointed many others who looked in vain for a mainstream party that would represent the values and principles that Labour seemed to have abandoned.

Most importantly, the move rightwards confirmed that most dangerous and insidious of Thatcherite platitudes – that there is no alternative – and left their natural supporters nowhere to go. Little wonder that the disappointment with nine years of New Labour left many voters disaffected in respect of democratic politics as a whole.

The outcome of the contest for power depends less on what part of the political spectrum each party occupies as on how well they address the particular issues with the greatest salience. The contest is therefore one of competing narratives; as the political agenda unfolds and throws up its usual bewildering array of issues, what matters is how well rival politicians can describe, explain and resolve them in terms of the values and attitudes that they are known or assumed to espouse.

New Labour possessed no such competing narrative. Because they had assumed that voters no longer retained traditional Labour values, no attempt was made to call them up – submerged as they might be – from the deep.

They therefore failed to link policies to deal with important issues with the values they were known to represent. They found themselves, for example, disabled from responding effectively to issues like widening inequality because they had lost the capacity to explain its significance in terms of most people’s continuing desire for fairness and for ensuring that both benefits and burdens are fairly shared.

They failed to engage with high rates of unemployment, or the downward pressure on wages, or the cuts in public services, by making the simple point that to cut purchasing power and government spending not only meant that the most vulnerable were asked to bear the greatest burden but also made no economic sense. This left the rival narrative from the right – that inequality is the price that must be paid for building an efficient economy – unchallenged. Yet what is economically efficient about keeping large numbers out of work or allowing wealth to concentrate in just a few hands?

When Tony Blair forfeited the trust of the people, New Labour accordingly had nothing left to offer. The failure of neo-liberalism has offered Labour a new chance. What they must now do is to stop looking for advantage by moving backwards and forwards along the political spectrum. They must renounce the constant triangulation and spin-doctoring, and develop those narratives that explain the country’s problems and their solution in terms of the values – fairness, compassion, tolerance, shared responsibility – that have long been held, have never been abandoned and that will again strike a chord in the minds and hearts of the people.

One of the fascinations of politics is that it unfolds over different time scales and at different levels. At one end of the scale are the personal and short-term; at the other, the matters of policy and principle, the history and development of ideas and of political movements.

It is no surprise – given the symbiotic relationship between the practitioners of politics and of political reporting at any given moment – that the latter should habitually focus on the human-interest immediacy of the movements in individual political (and other) fortunes as they swing up and down. A case in point, last weekend, were the reports detailing the latest turn in the career of Peter Mandelson.

The on-again off-again career of Lord Mandelson has provided much innocent entertainment for observers and a rich seam of copy for political commentators over the years. Yet, even so, it is surprising that the momentous events of recent weeks, which could herald a seismic transformation of the British political landscape, could have been viewed through such a narrow lens as was seen in the articles about Lord Mandelson’s latest transformation.

It is true that Andrew Rawnsley ended his piece by briefly taking a wider perspective. The story was not, we were solemnly assured, one of unalloyed triumph. Peter Mandelson, we were told, genuinely cared about the Labour Party; its probably imminent demise was enough to turn the moment of his greatest success into a personal tragedy.

We can readily agree that the demise of the Labour Party would be a tragedy – but surely a tragedy on a much greater scale than of one individual’s personal disappointment. It is doubtful, after all, if many tears will be shed for Lord Mandelson. Many – including all those whose allegiance to the Labour Party over recent years has been sorely tested, as well as those who have rejected Labour in favour of other promises to defend their interests – will see the noble Lord’s disappointment as being richly deserved.

This is not a tragedy in the Shakespearian mould – a fatally flawed individual being undone by his inability to deny the power of the flaw that drives him. This is a tragedy that is likely to engulf an audience of millions, not just the leading members of the cast.

Peter Mandelson is rather in the position of a ship designer whose vessel is revealed to be unseaworthy. He is consoled by observers with the assurance that they know that he did not mean it to ship water and, having arranged a lifeboat for himself, he then persuades the captain to stay on the bridge until the ship goes down.

New Labour was, after all, Peter Mandelson’s project par excellence. He signed others up for the journey, and was initially fortunate enough to engage a brilliant skipper for the project. But, when a new captain proved to be no seaman, and the ship’s design faults meant that it foundered, disappointment is hardly an adequate sentiment. Those who entrusted their lives and life-chances to the seaworthiness of the vessel are entitled to require the designer to accept responsibility for, as well as disappointment at, the loss – and at the tragedy that is theirs.

We are surely now in a position to judge the New Labour project, not according to the claims of its progenitors, but in the light of its likely final outcome. The project started, after all, with an overwhelming parliamentary majority, a popular and telegenic leader, a huge appetite for change and a real sense of excitement and anticipation in the country – all delivered by an overwhelming election victory.

What has happened in the twelve short years since then is not the familiar story of a government that gradually loses support and – having used its opportunity to best effect but nevertheless having been exhausted by its efforts – is defeated by newly revitalised opponents. The immediate future facing the Labour Party is one of virtual extinction as a party of government. It may take a generation for Labour to recover – if it does so at all. In the meantime, the effective voice of the democratic left – the most consistent and reliable generator of change and reform we have – will be stilled.

That is the real tragedy – not to be expressed in terms of individual careers – but in the destruction of one of the main forces in our democratic politics. Without it, our whole politics will be poorer. Faith in our democratic institutions and processes will be weaker.

It is an outcome, a tragic outcome, that is the inevitable and predictable consequence of deliberately removing Labour from its value base and from its supporters. It is the direct result of treating power as an end in itself, of seeking power for the purpose of simply perpetuating it. New Labour not only failed to take a once-in-a-century opportunity; it turned its back on the idealism and creativity that, under President Obama, is reinvigorating American society and politics. This is the end result by which Peter Mandelson should be judged. You bet it’s a tragedy.

So, it has come to this. In twelve short years, New Labour has travelled from the exultation and boundless optimism of the 1997 election victory and of a movement whose time had come to a craven pursuit of self-preservation by frightened MPs.

And even then, it is a self-preservation that can only be pro tem. So narrow have horizons become, so short-term the perspective, that the interests of party and country will be sacrificed for a few more miserable months in a crippled parliament and a dying government.

Let no one be deceived into thinking that the decision to soldier on under Gordon Brown’s leadership was taken in the wider interests of the party or the country. Nor was it a signal of affection or respect for their leader, or even just loyalty. This was every man for himself. The only calculation that seems to have mattered was the one that said that the pay packet could be kept coming for a few more months. For who could doubt that the warning from Lord Mandelson, with his unerring instinct for the baser motivations of political life, that to change leaders would inevitably mean an early general election, was enough to stop many potential rebels in their tracks?

This, then, is the end game of the New Labour project. To the extent that history ever makes final judgments, we can begin to see where it has led – not to a long period of Labour hegemony, as was so confidently foreseen, but to the real danger that Labour will have destroyed itself.

Let us remind ourselves of the course pursued by New Labour strategists. The catalyst for that strategy was the 1992 election defeat, although its seeds probably go back to 1987. The Mandelsons, Blairs and Browns concluded in the wake of those defeats that Labour was unelectable and that the Thatcherite hegemony could not be successfully challenged. They decided that the greater part of Labour’s analysis of what was needed to reform Britain should be abandoned, and that the Thatcherite agenda should in essence be adopted.

In this, they were surely wrong. Of course Labour needed to modernise, and to adapt its principles to new and continually evolving challenges. But the Thatcherite revolution had largely run its course. Mrs Thatcher herself had been rejected by her own party, which proceeded – under John Major and his successors – on an increasingly uncertain course. By 1997, the Labour alternative under almost any leadership would have defeated the Tories. The sacrifice of Labour’s central values in favour of a callow and unsophisticated acceptance of the market’s infallibility was simply unnecessary.

By then, however, the New Labour style and purpose had been fully developed. The project developed its own ideology. What mattered was the winning and keeping of power, rather than actually doing something with it. Power, once achieved, should be used for its main purpose – to perpetuate itself. New Labour would be all things to all men, taking the pain and hassle – and even the politics – out of politics. It would occupy the centre ground, careful not to offend the powerful. It would thereby force other contenders to the margins, and usher in a long period of unchallengeable dominance.

The simplicity of this goal and this strategy meant that the Labour Party itself could largely be ignored, both as an organisation and as a source of ideas and analysis. New Labour leaders and tacticians could appeal directly to the voters, through the media, and through spinning the message, and could thereby free themselves from the need to take the party with them. The loyalty of the ambitious could be guaranteed since they would quickly recognise that the path to power lay through New Labour.

It hasn’t quite worked out that way. The voters quickly tired of spin. They intuitively recognised the unattractive limitations of pursuing power at any cost. They were repelled by the contortions produced by the absence of principle and strong values. But most of all, they were brought in time to make an ever harsher judgment of policy failures.

Those failures were partly the result of hubris – on the part of a leader who was so persuaded of his moral infallibility that the country could be led into an ill-judged invasion of another country on no greater foundation than his own say-so – and partly the consequence of a sickening obsequiousness in the face of the rich and powerful. And, while New Labour could just about be held together by a brilliant front man, there was nothing else to fall back on when the voters tired of him and when his successor was revealed as totally lacking in political – not to say human – skills.

So, the voters look certain to reject New Labour. A whole generation of Labour leaders who could and should have stood for something more than simply hanging on to power will close the New Labour chapter by – appropriately enough – doing just that for a few more months. It will be a long and hard road back if a renewed (please not “New”) Labour Party is to rise from the ashes.

As the Labour Party steels itself for electoral meltdown, it may seem ironic – after the global-sized catastrophes of the Iraq invasion and the worldwide recession – that it is the descent into venality at home that will count most with the voters. But to underestimate the importance of the expenses scandal would be a mistake.

The voters understand intuitively that, having presided over and applauded a society in which greed and the pursuit of self-interest have been elevated into positive virtues, New Labour’s own pursuit of power at any cost has produced its inevitable outcomes. The expenses debacle has been much more than a series of individual peccadilloes and defalcations; it has been the expression of a political culture that has created a gulf between what is seen as acceptable and necessary in the political world and the standards of decent behaviour expected of the rest of us. The individual manifestations of that culture may seem grubbily petty and venal, but the embarrassed squirming among the political class as the detail has been exposed is testimony to how out of touch our leaders had become and how serious that is for the whole political process.

This matters more to Labour than to others. The Tories have never bothered to hide their view that power is to be sought so that it can be used to defend vested interests. The Liberals seem to believe that power is best exercised by “nice” people. Only Labour, traditionally, has pursued power with the avowed purpose of correcting the unfairness and inefficiency of allowing the dice to lie where they fall and of creating a better society.

It is for that reason that the demise of Labour – under its “New Labour” leadership – is a matter not just for pain and anger at the loss of the opportunity presented by the 1997 election victory, and contempt for those who led us down this cul-de-sac into disreputability. It is also a major blow to our whole political structure which, in the absence of a substantial presence from the democratic left, will be less effective at creating a healthy society and a strong economy than it should be.

The special importance of the left lies not just in the fact that it is, or at least has been, the major source of progressive ideas, that it has provided the most reliable stimulus of new thinking, that it has generated the most creative dynamic for reform – though all of that is true. Its true value is that it underpins the whole case for democracy and for the power of good government.

Among the many lessons we should draw from the global recession is that this is what happens if government fails in its purpose. Ever since democracy was ushered in, there has been no shortage of powerful forces dedicated to undermining it. This is for the obvious reason that the whole point of democracy is to offset the power of the powerful with the political strength of the people. In the absence of that political power, without bringing to bear the legitimacy of the democratic mandate through an elected government, there is no force capable of resisting the might of the economically or socially or militarily powerful.

The failure of government to lean against the economically powerful over the last three decades led directly to the unregulated excesses that created a market-driven recession. And, even as we grapple with the measures needed to recover from recession, the same central question is starkly posed – what is the proper role of government?

The key feature of a recession is that every individual, every business, will have a cast-iron and rational reason for battening down the hatches. Only government has the capability and responsibility to act in a contra-cyclical way, against market logic, and to pull us out of recession faster than would otherwise happen, by spending and investing at a time when no one else will.

What this tells us is that it is always the role of government – when necessary – to represent the wider interest against powerful forces, and to act in a way that would be irrational or impossible for the private individual, however powerful. It is only the left that has in the past carried into government this central concept of what the true purpose of democratic government really is.

If this week’s elections do indeed show how thoroughly New Labour has debased and betrayed the legacy with which it was entrusted, it will not just be Labour’s party warriors who are relegated and enfeebled. The vast majority of the British people – irrespective of their party allegiances or lack of them – will have been significantly disenfranchised. The blow struck by the expenses scandal against faith in the democratic process will claim more casualties than just a few MPs. The real losers from the demise of the Labour Party will be millions of ordinary people who – perhaps without knowing it – will have lost their best defence against the depredations of the powerful.

As the scandal of MPs’ expenses unfolds, it threatens not only to expose individual sleaze but to create a crisis of confidence that will engulf the whole of the political class and Parliament as well.

While it’s not surprising that Labour, as the party in power, is in the firing line, the Tories, and others, all have their skeletons in the cupboard too. But for Labour loyalists, it is not enough to say that others are equally culpable; we feel that we have the right to expect better from Labour.

It is probably not too harsh to say that we don’t necessarily expect too much by way of principle from the Tories. They have always been a party of the self-interested. That self-interest is sometimes – in their better moments – tempered by a touch of noblesse oblige; but, in these days of the “self-made man”, there hasn’t been too much noblesse in evidence.

Politics is, we know, a difficult business which demands compromise and the adjustment of principle to suit reality; but Labour politics has always seemed to enjoy – even in an era of aggressive individualism – the redeeming influence of a genuine concern for others and for the well-being of the whole community. So, how did we come to this? How did Labour embrace a culture of self-aggrandisement, pursued even against the interests of the disadvantaged in that society that we like to insist, pace Margaret Thatcher, does exist?

It might be argued that we can’t indict the whole Labour Party because of the defalcations of a few. Well, that is indeed the question. Has New Labour’s “intense relaxation” about the “filthy rich” now been extended to ourselves? And does that indulgence cover the “filth” as well as the riches? Is this what New Labour now stands for in British politics?

Is there, in other words, a recognisable connection between the politics of New Labour and the fall from grace of individual Labour standard-bearers? The first evidence of this was the emphasis placed by the New Labour project on doing whatever was necessary in order to win power, even if that meant the abandonment of principle. Opinions may differ on whether the junking of much that had been considered to be core Labour values was really required for electoral victory – and no one can doubt that the way in which those values should be applied to the issues of the twenty-first century was overdue for re-appraisal.

But what was surprising about New Labour was the enthusiasm shown, not just for change and renewal, but for the positive adoption of a quite different agenda – one that had hitherto been seen, with its acceptance that the market should not be challenged and that growing inequality was the necessary condition for economic development, as the property of the right. The Labour Party found itself cut adrift from its traditional emphasis on the central role of government as defender of the weak, as a counter-force to an unfettered market and as a guarantor that everyone shared in growing prosperity.

As it floated free from its traditional moorings, little wonder that a new generation of Labour leaders became confused about what they were in politics for. If policy dictated that unashamed greed was indeed the irreplaceable mainspring of economic advance, how could it be wrong to act on those same precepts in one’s own life?

The whole thrust of the Blair government was, after all, that politics didn’t really matter, and indeed were best eschewed altogether. The Blair pitch was always that, if voters elected the right people (“pretty straight sort of guys”), they could safely forget about politics which would become nothing more than an annoying distraction – the domain of a few fanatics. The Labour Party was assured that it did not need political analysis or a programme for real reform.

But without that analysis and programme, what was power for? The question matters little to the right; power for them is the means by which the pace and direction of change can be controlled and, at times, completely frustrated. But for the left, power is surely a means to an end, to a different and better society – one which shares its benefits with everyone.

But that, too, was denied by New Labour. For them, the purpose of power was not to use it but to enjoy it and extend it. Power was a state of being, not a path to change. The over-riding priority of New Labour was always, from the first day of taking power, to retain it by winning the next election. Power as the instrument of change would be limited to those measures that did not alienate powerful interests and thereby jeopardise the perpetuation of power.

That is why controlling the agenda through spin, why manipulating events through a mastery of the minutiae, of the often grubby day-to-day detail of politics, became the leit-motifs of New Labour government. They rarely bothered to lift their eyes to wider horizons or to re-connect with their core values. Tomorrow’s headlines were always their prime concern.

Little wonder, then, that New Labour leaders lost their way. Short-term advantage, politically and personally, was all that mattered. The Labour Party is paying a heavy price for that distorted view of what left politics should be about.

Bryan Gould

12 May 2009

This article was published in the online Guardian (Comment Is Free) on 13 May