Falling membership has allowed small cliques to control our politics. It’s a failed model, but the powerful like it that way

A constituency party is reported to be in a state of “civil war”. Thirteen experienced councillors, including past mayors and cabinet members, are said to have suddenly been “removed” from candidate lists. There are allegations of people voting in selection contests on other people’s behalf, and telling lies about why councillors have apparently been dropped. “It’s a stitch-up,” says a councillor who has served his party since 1978; it’s all about “revenge”.

On the basis of the Labour party’s current problems, you’d probably expect all that to be happening deep in the red heartland: post-industrial Scotland, the south Wales valleys, inner-city Birmingham. But no: this is a story engulfing the Conservative party in Romford, which involves council candidates for the outer east London borough of Havering. There, the Tories have 33 seats to Labour’s five, with the rest held by independents. As with Falkirk, it is exactly the kind of place where a mixture of loyal voters and shrinking party membership has apparently created an ideal setting for internal skulduggery. Control the moribund local machinery, it seems, and power will be yours.

Labour’s crisis, we are told, is down to its relationship with the big trade unions. And yes, the Falkirk imbroglio – which may yet spread to other places – does highlight a mess of problems traceable to the state of the so-called union link: the emergence of huge “super-unions”, the arcane rules governing their role in the party, and more. But the fundamental issue runs even deeper: it is about the broken-down state of our political parties, and the fact that it has long been in the interests of powerful forces to leave them exactly as they are.

The Tories have their own versions of these episodes, usually involving small groups of irate local activists, and manipulative moves by the leadership: in such places as Surrey East and Westminster North, we have seen much the same plot lines. The point is, the main political parties are dying fast – which now leaves them open to capture (witness a killer line from an internal Unite document about Falkirk: “We have recruited well over 100 Unite members to the party in a constituency with less than 200 members”). And unless they start to change, democracy itself will be just as damaged.

For sure, the demise of traditional political activism has been accelerated by the contempt in which it has been held by both Blair and David Cameron – which may partly explain why, at a mere 1.1% of the population, the UK’s proportion of party members is among the lowest in Europe. But clearly, much bigger forces are at work. If they are interested in party politics at all, people increasingly have plural identities. At a National Union of Students conference before the last election, I met a young woman who had simultaneously joined Labour and the Lib Dems, keen to get the measure of both, not knowing that if either found out she’d be expelled. My own tribal affinity, for all that it often fails to pass the test of basic rationality, is still with Labour, but I have canvassed for the Lib Dems (in an attempt to keep the Tories out – I know, I know) and voted Green. In a world where people dip in and out of groupings with the ease facilitated by the “Like” button in Facebook, this is what the future will look like – and the closed, tightly disciplined model of political parties will increasingly seem downright absurd.

At the same time, the idea that you should join anything that denies you a meaningful say looks equally ridiculous. The Tories have never been the most democratic of parties; from the start of the Blair era onwards, Labour has moved to much the same place. Unless you are part of some Unite-esque scheme to join up as part of a grand revolutionary plan, why would you bother shelling out for a membership card?

Annual conferences are an expensive joke; when it comes to policy, the people at the top are usually a law unto themselves. That only leaves the election of leaders, a ritual that periodically creates small membership surges, before they quickly fall away. Primaries involving registered supporters might be part of the answer, if only as a means of pulling people into parties’ orbits – but unless they fit into a bigger project of collective empowerment, they could just as easily feed the power of the current elites.

Parties, then, need to somehow be both looser and more democratic. But to be loosely organised is often to be dominated by shifting elites, who simply expect members to stuff envelopes and wave flags (witness the US). Democracy necessarily entails a defined body of people. Can all of this be resolved?

The change will have to be as much cultural as organisational. Compass, once a Labour-aligned group, has put one foot outside the party and recruited Greens, Lib Dems and people with no party ties at all. It has begun to explore a future in which people from distinct groups might better achieve their ends by breaching the party system’s oppressive walls.

A rather panicked Ed Miliband is to make a speech about all this on Tuesday. Though it looks like the time has come to move to an opt-in rather than opt-out model for individual union members’ relationships with his party, he is right to be resisting those idiot voices urging some formal uncoupling of the unions and Labour: if millions of working people do not have a dependable means of accessing mainstream politics, capital will have scored another win – and our democracy will be even more impoverished.

Overall, Miliband should be judged by whether what he says feels like an appreciable step away from a failed model, or nervous tinkering with a status quo that points in only one direction: the kind of events witnessed in Falkirk, Havering, Blaenau Gwent and Westminster North extending into the distance.

Labour is leaving the political foreground dangerously empty, which is also why the Falkirk brouhaha has acquired such clout

So, there goes Tom Watson, seemingly in an attempt to allow the Labour leadership to put paid to an increasingly tangled controversy. Its component parts run as follows: the Unite union; its alleged attempt to cook the selection of the Labour candidate in Falkirk; the fact that the union’s favoured hopeful in that seat is a close Watson aide; claims that Unite has its sights on up to 40 other seats; the fact that the union’s leader, Len McCluskey, was once Watson’s flatmate; and more.

Much of this is soap opera of the kind that can be found in spades in any political party. But what makes it so highly charged is the fact that it speaks volumes about a left/right faction-fight that now seems to be reaching its peak – hence the line in Watson’s shadow cabinet resignation letter that claims that “it is better for you [Ed Miliband] and the future unity of the party that I go now”.

Watson, let us not forget, was campaign co-ordinator for the next general election – an increasingly impossible role to carry out when he was a) so associated with Unite and its quest to tilt the party towards the old left, and in any case, b) the object of an animus from the right of the party that goes back to his role in the departure of Tony Blair.

The heat that Watson has attracted has only ever been a symptom of two underlying problems, which remain.

First, there is the fact that the Labour leadership and shadow cabinet are not generating nearly enough noise, and thereby creating a media vacuum. At the moment, Miliband tends to make speeches that may or may not attract much attention, as well doing PMQs and occasionally piping up on whatever issues are deemed to deserve his attention. Many of his colleagues are borderline mute. The space is filled by an increasingly surreal pantomime that is a laugh to write about, but has long since drifted away from anything remotely real.

Fair play to Jones and Hodges: they are both very readable voices with much to say about the state of modern Labour politics. But if they are getting more attention that most shadow ministers, what does that tell you? Answer: Labour is leaving the political foreground dangerously empty, which is also why the Unite brouhaha and every utterance from Len McCluskey have acquired such clout.

Second, there is the fact that a row over selections between the Blairite right and the Unite-led left says a huge amount about Labour’s moribund model of organisation. If constituency Labour parties are open to abuses from those factions of the party – the Westminster-endorsed parachute in the case of the former; the packing of local memberships when it comes to the latter – that’s simply more proof that Labour cannot go on with a dwindling membership, and processes that tend not to reach beyond the walls of damp meeting rooms on Wednesday evenings.

Sooner or later, the people in charge of the party will have to face facts: the orthodox mass party is a dead idea, and Labour will have to reorganise itself and be reintroduced to a pluralistic, politically sophisticated world. A party machine essentially run along the same lines as it was before the second world war no longer cuts it: the Falkirk controversy is merely the latest evidence. On Tuesday, I advocated open primaries as a way out of this mess: a remedy also proposed by the Times and that great leftist Daniel Hannan MEP. It’s the right solution.

Meanwhile, we await some decisive word on all this from the Labour leader himself. It’s pretty obvious that any words – no, action – need to be bold, loud and imaginative, not least because the absence of such qualities in current Labour politics is part of the reason the party has ended up in this mess. And so to the obligatory conclusion: naturally, don’t hold your breath.

Labour’s bitter feud with Unite over goings-on in Falkirk shows that both the party and union are out of touch

What a sorry mess. If you’re one of those entirely rational people who tend to avert their eyes from stories about the internal workings of the Labour party, you may have so far missed the brouhaha about candidate selection in the Scottish constituency of Falkirk. The party’s high command has suspended the troubled selection for a parliamentary candidate there, after allegations that the trade union Unite has been signing people up as party members en masse, supposedly without some people’s actual knowledge. Its aim, it is said, was to secure the Labour candidacy in 2015 (the sitting MP is the disgraced former army major, Eric Joyce) for Karie Murphy, a senior aide to the party’s head of campaigning, Tom Watson, and the favoured candidate of the union’s leader, Len McCluskey – who, some people are fond of pointing out, was once Watson’s flatmate.

Murphy has now stood aside, but the row about Falkirk goes on. Kim Howells, the former Labour minister and MP for the Welsh seat of Pontypridd, has piped up and, given that part of the world’s association with a somewhat closed, anti-democratic kind of Labour politics, he should know. He thinks Unite’s alleged behaviour “threatens the whole reputation of the Labour party”. So does Peter Mandelson, who said that episodes such as the Falkirk debacle “risk damaging Labour’s reputation and undermining our electoral appeal“.

Meanwhile, Unite is threatening legal action against the party to which it is by far the largest donor, and pointing out that its championing of such candidates as Murphy is part of a drive to “ensure that people who share working class and trade union values are successfully selected as Labour party prospective parliamentary candidates in winnable constituencies just like Falkirk”.

The kind of shenanigans said to have occurred in Falkirk is as much a part of Labour history as any of its more romantic aspects. In 27 years as an on-off follower of the party’s internal affairs, I have heard the same stuff time and again: whispers about ghost members, signed up in their droves and paid for with a single cheque; talk about parties within parties, and conspiracies to fix selections; cases in which favoured candidates have been parachuted into seats – and, shall we say, assisted by powerful forces in the party’s upper echelons.

Most political parties have their own versions of such stories, but the Labour one is particularly entertaining. The Blairite right has been as involved as the union-backed/Brownite left: witness the great stink kicked up in Blaenau Gwent circa 2005, the short-lived kerfuffle about Georgia Gould, the daughter of Tony Blair’s polling guru Phillip Gould, or the disaffection and anger sparked by suggestions that Mandelson – yes, him again – had effectively reserved the seat of Stoke Central for his good friend Tristram Hunt.

There are some days when the entirety of Labour’s build-up to the next election seems to be reducible to an ongoing dogfight over candidacies between Unite, and the Blairite pressure group Progress. The former, to hear the latter talk, are antediluvian thugs who intend to turn the party into McCluskey’s fifth limb; from the reverse perspective, Progress is the vanguard of an entryist conspiracy by Oxbridge-educated neoliberal fifth columnists (or something). Both apparently believe that they are on the side of righteousness. And, I suppose, that the ends justify the means.

Which brings us to a great Labour mistake that is being exposed as never before. Contrary to centre-left groupthink down the ages, the ends do not justify the means. In fact, if you conduct yourself in a closed, anti-democratic, fix-crazed kind of way behind the scenes, you will poison your own politics to the point that any talk about being “progressive” will start to look very questionable. By the end of his time in office, the irreconcilability of those two elements was etched all over Brown’s face.

The same tension was there in the Blairites’ cynical use of Old Labour political machines to bag the required number of safe seats (check out how many senior New Labour figures had seats in the north east). And, as much as it pains me to write this, it seems to be evident in the big unions’ apparent attempts to further the course of working-class emancipation via the use of stitch-ups. Gandhi had it right: you have to walk it as you talk it, and “be the change you wish to see in the world”. Labour has never really understood that notion; perhaps it never will.

Is there any way out? There actually might be. Some time ago, I wrote a piece for the Guardian opposing the idea of open primaries for candidate selections. I said they would increase the influence of big money, and reduce politics to the stuff of name-and-face recognition, as well as diverting attention from the case for electoral reform. I still have my misgivings. But if the Falkirk story – along with others – tells us anything, it is that the standard-issue Labour model of candidate selection is broken, and that something has to be done to prise the process away from entrenched interests who are trying to sustain a grim kind of politics-by-clique. In the modern world, they won’t get away with it: news can now get out in a flash. At the same time, however, orthodox political parties are apparently shrinking into moribund husks, which only increases the temptation to stage takeovers, and fix internal processes.

What we need is a great opening-out in all the parties, via which genuinely local voices and maverick candidates can come to the fore, and the fabled parachute is folded away. The best argument in favour of primaries, it seems to me, is the presence in the Commons of the Conservative MP, Sarah Wollaston, selected via an open system (which the Tories now seem to have shunned), and precisely the kind of uncontrollable, expert, irreverent voice that so rarely comes to the fore. She is arguably parliament’s most interesting presence. Such is the challenge Labour faces, but seems barely interested in addressing: to either get with a kind of politics that sits comfortably with the 21st century and might demonstrate at least some belief in the practice of democracy; or to leave things unchanged, while the intrigue and scandal extends into the distance.

• Young people are supposed to be left-leaning idealists, but polls tell us that today’s under-34s don’t believe in handouts and high taxes – and they’re voting for David Cameron. By John Harris. Click here to read article

• 160 years ago a giant sequoia in California was cut down, becoming the inspiration for the national park system. By Leo Hickman. Click here to read article

• Faced with scarcity of British contenders at Wimbledon, The Lawn Tennis Association must prove it can build from the bottom rather than the top. By Kevin Mitchell. Click here to read article

In our audiobook review this week we seek out thrills with Steven King’s Joyland, and The Kill Room by Jeffrey Deaver.

The Guardian Audio Edition is supported by Audible.co.uk. To listen to the audiobooks reviewed in this week’s edition go to audible.co.uk/guardian audio.