'Blair and Thatcher tried to crush me. Both failed'

Four years ago he was expelled by Labour for daring to stand as mayor of London. Tomorrow, he launches his manifesto for a second term, after being enticed back by the party that had condemned him as a failure. How did he do it? Hugh Muir on the remarkable renaissance of Ken Livingstone

Five months ago, just days after being readmitted into the Labour party from which he had been expelled four years previously, Ken Livingstone, fuelled by just enough alcohol to make him merry, sidled over to a small group of party colleagues. His giggles cut through their earnest chatter. "There have been two dominant political figures of the last two decades," he said. "Blair and Thatcher. Both have tried to crush me - and both have failed."

There is much that explains Livingstone in that anecdote. The joie de vivre that leads him to drink a little too much; the capacity for indiscretion which allows him to share such irreverent thoughts with others; the ego that requires him to trumpet his triumph. And, of course, the fact that, stripped of bravado, the comment is broadly right.

These are the happiest of times for Livingstone. Four years ago, he romped to victory in the London mayoral elections as an independent, despite the Labour skulduggery that had robbed him of their official candidature, and then sought to squash his rebel campaign. Tomorrow, he launches his manifesto for a second term as mayor, ahead of the June 10 elections, hoping for another resounding victory.

If it happens - a second term is far from assured, but he remains the favourite - it will complete a quite remarkable political renaissance. Livingstone will be Labour's figurehead in its most important region. He will expect to have access to the party's machine, and partial stewardship of some of its most cherished social initiatives, many of which are being developed in London. For supporters such as Mark Seddon, the editor of Tribune, a member of Labour's National Executive and a long-time Livingstone backer, it is down to the veteran leftwinger's personal guile. "Ken outfoxed Tony Blair," Seddon says. "He is very good at playing them at their own game."

How exactly did he do it? Was it thanks to his policies or his personality? Did Labour change or did he? Is he still the "Red Ken" of tabloid fame - was he ever? And perhaps more importantly, can he keep his apparent winning streak going?

There is a small circle of traditional Labour red in the background of Livingstone's election poster, but more prominent is another circle in purple: his campaigning colour. Emblazoned across them both is the slogan: "Vote Ken on June 10." Close observers may notice, in the bottom right-hand corner, the word "Labour" in small type, but most will probably miss it. These posters were plastered all over the Avenue's Youth Project in Queen's Park as Livingstone, Blair and the party's European and London Assembly candidates launched the campaign earlier this month.

The prime minister, all smiles and palpably uneasy, was applauded as he entered the room "I always said Ken would make a great mayor," he joked. Then, more seriously, he added: "People ask me why I changed my mind, and the reason is because I was able to judge the record that he had."

Blair's words bore witness to the multi-pronged strategy Livingstone has pursued since being handed a second chance to run London four years ago. Stepping into the role, he indicated that he would seek to resume what he had begun while leader of the Greater London Council, until the Thatcher government, threatened by the GLC's radical agenda, abolished it in 1986. His first words to acknowledge the win were: "Now, before I was so rudely interrupted 14 years ago ..." They triggered wild applause.

But if he did not admit it then, Livingstone recognised that his only option was to demonstrate that just as times had moved on since the abolition of the GLC, so had he. Some of the ideas survived the interregnum - notably, so did some of the faces - but Livingstone has had to work hard to challenge the widely held belief that he was a political dilettante, high on showy rhetoric but low on delivery.

"I think he is just more aware in this millennium about what gives him appeal as a politician," says Andrew Collinge, the head of local government research at pollsters Mori. "He has always been a very good politician, but now he is aware of the fact that it is better to try and please more of the people more of the time. In the old days he was content to be a fly in the ointment." Karen Buck, chair of the London Group of 54 Labour MPs, sees a man whose sense of self-worth always required him, eventually, to hold meaningful power. "I think he always wanted to run something," she says. "Before, all we had was 'Lippy Ken'. He was frustrated on the backbenches." One government minister, who dutifully booed while Livingstone was persona non grata within the party, is now openly admiring of the mayor's achievements, especially his audacity. "The congestion charge changed everything. Having the nerve to push that through is what made the party take him back."

The congestion charge is Livingstone's most obvious first-term achievement. Londoners have now become accustomed to the charge, which Livingstone introduced three years after coming to power. Up to 70,000 fewer cars now travel into central London each year. Most routinely pay the £5 charge; later in the year they will do so by direct debit. The whole process will become virtually automatic. Not everyone, of course, has welcomed the innovation: major retailers complain that the charge scares off shoppers - John Lewis claims its sales are down by 5%. Lobby groups in west London oppose any suggestion that the zone should extend towards Kensington. Others point out that Capital, the private firm charged with administering the tax, received millions to operate a penalty system that underperformed - the mayor is reputed to have spent £75m just to keep the company working efficiently. The crucial issue, however, is that congestion charging is in place and no one can continue to argue that road pricing cannot work. If anything, the complaints suggest it can work too well.

As he attempted to work out, four years ago, what could be done to solve London's traffic problems, Livingstone found that all roads led to a thickset, bespectacled engineer called Derek Turner. As traffic director for London, Turner had been responsible a decade previously for the red routes programme that identified 360 miles of key routes throughout London to be enforced as non-stop thoroughfares. That idea caused howls of protest at the time but the bullish Turner had pushed it through.

He first raised the notion of road pricing in London in the early 70s. Livingstone had promised in his 2000 manifesto to introduce a congestion charge, but it was not clear that he was serious - the other major candidates, before the election, had sought out Turner for detailed briefings about how such a scheme might work, but Livingstone didn't bother. He simply took details from a discussion document and decided to proceed. "We had a three-hour meeting and he said road pricing had always been his general philosophy," Turner recalls. "He would not go for workplace parking because the business community didn't like it. He made it clear he wanted it delivered in the first term and that all he wanted was the simplest scheme."

The charge was certainly not universally welcomed. The Tories strongly opposed it while newspapers said it was unfair and would cause gridlock on London's outskirts. Even within City Hall, many of the mayor's closest advisers doubted the necessity of such a grand gesture. But Livingstone, perhaps aware of what was personally at stake, remained determined to press ahead. "There was one occasion when Ken pointed to me and said, 'If this man has a heart attack, I am finished,'" says Turner. "But he never once suggested that we should stop." On February 18 2003, the switches were flicked and the new system launched without a hitch. The mayor was joyous. "Derek, you've won me the next election," a beaming mayor told Turner. "But you have done it a year too soon."

In the run-up to the charge, Livingstone and his team were forced to endure controversy about the major redevelopment of Trafalgar Square, and the effect that project had on traffic in London. With an ambition some viewed as almost suicidal, the mayor's team had simultaneously embarked on major roadwork projects at Shoreditch in the east of the city and Vauxhall Cross in the south. It meant that three major routes in and around central London were practically no-go areas. The Trafalgar Square initiative alone necessitated the adjustment of 25 sets of traffic lights to control the flow of cars entering the square.

Masterminded by Lord Foster, the changes at Trafalgar Square have, in fact, been one of Livingstone's biggest successes, placing one of the capital's principal landmarks, in Livingstone's words, "on a par with the great central squares in Europe". Traffic has been re-routed from the north side and the road has been replaced with a grand piazza. A staircase has been built from the terrace in front of the National Gallery down into the square itself. More buses pass through and the public space, which was cleared of pigeons, is regularly used for his major public events.

Turner viewed the plans in engineering terms, and recommended that the redevelopment should take place after the introduction of congestion charging, when traffic levels were lower. But to the mayor's office and Transport for London, the capital's transport body created alongside the GLA, that was bad politics. They preferred to do all the work at the same time so when the charge was introduced it would have an uninterrupted period to bed down. Inevitably, the simultaneous redevelopment and rephasing of traffic lights triggered gridlock, a media backlash and a political row at City Hall. By the time the storm blew over, relationships between Livingstone's officials at TFL and Turner had been irrevocably damaged. Turner left to become a private consultant developing possible congestion charge schemes for cities around the world. When the man who gave Livingstone his most high-profile triumph left, the mayor's aides shed few tears.

Steve Norris, the Conservative mayoral candidate in 2000 and this year, has said that Turner was "stitched up" and that he will offer him a job if he wins the mayoralty. But ultimately, it is Livingstone who has claimed the credit for the innovation. "He has made the argument for having a mayor," says Diane Abbott, the Labour MP who has served on Livingstone's advisory cabinet. "He made the decision to do something innovative that an ordinary government or council setup would not have had the courage to do."

Senior members of the government reached the same conclusion, hence the decision to invite Livingstone back into the party - although according to one Blairite minister, his return remains a difficult issue for some. "He is a titanic figure. He is either loved or hated, and there is group of people that really hate him. There are the people who were close to Kinnock and who still have strong feelings about militants and the leftwing. They will never like him and they are pissed off that he is back.

"Another group within New Labour, including the prime minister, are more pragmatic and they respect a politician who is very capable - and Ken is without doubt one of the most capable politicians of his generation. There is also, of course, political expediency. They understand he is a vote winner." There is a grudging admiration, too, that he has thrived under a system that was specifically designed to paralyse him. "He hasn't many powers but he has used what he has quite cleverly."

"Mrs Thatcher used to say that when I see a man of 25 riding the bus, I see a man who has failed in his life," he told the London Assembly. "That is the kind of view we are going to change." The result was the expansion in the bus service, the second "big ticket" achievement of his first term.

Last week the number of people using the buses in a year in London hit six million, the highest total for 40 years. Since 2000, bus use is up 33%. There has been a shift of people from cars to buses of more than 4% in four years, allowing the Confederation of Passenger Transport to boast last week that the number of bus journeys across the country rose by 110m last year. There are 7,000 buses in the city's fleet, 1,200 more than there were four years ago, and 4,500 have been replaced.

The mayor, having been denied control of the underground by national government, realised early on that he had to focus on the buses in order to seek re-election, and hired Peter Hendy to head the initiative. Hendy had joined London Transport as a graduate trainee; as part of the team that bought Centre West buses from London Transport, he made a £4m personal fortune when the company was sold to First Group in 1997. Despite his ideological distaste for privatisation fat cats, Livingstone decided Hendy could be his very own poacher-turned-gamekeeper. "He is the perfect person," he said. "He knows all the scams."

"When Ken took office one of the things immediately apparent was that because of the fuss over PPP, he would not get the underground on day one, and that his influence over the railways was small," says Hendy. "Only the bus service was capable of intervention. We decided that the buses were being run at minimum cost rather than for maximum public benefit."

With the help of government subsidies they were able to tear up many of the prevailing rules for running privatised, competitively tendered services. New "Quality Incentive" contracts were drawn up that pay the bus companies special bonuses for running reliable services, and fine them if they do not. Bus companies were told to run more buses where survey evidence suggested they were necessary, and were paid accordingly. Crucially, the service provided mattered more than price: only half of the contracts went to companies who bid the lowest. At the same time fares were slashed from an average cost per journey of 54p to 45p. The congestion charge, by deterring car use, quickly pushed up the number of passengers. Amid some suspicion among the privatised firms about the mayor and his motives, it was Hendy who negotiated with the bus companies. "I said to them, 'Here is a fantastic opportunity'," he says. "If you don't want to take it or don't think you are up to it, we will go somewhere else."

The other significant figure in Livingstone's transport programme - and in his rehabilitation - is his American transport commissioner Bob Kiley. Two of the three "big ticket" achievements of the mayor's first term have emerged directly from Windsor House, the imposing office block in Victoria which is Kiley's fiefdom.

The mayor has already confirmed that if he wins, Kiley will be offered a second contract. His first was worth £2m. He will also continue to live in the £2m house Transport for London bought for him in central London - the most expensive municipal property in Britain. (Stung by accusations of excessive largesse, the mayor's office once claimed it was in relatively downmarket Pimlico; it is, in fact, in Belgravia.)

Kiley should have come to London as part of a package. Although the American is widely credited with personally transforming the fortunes of the New York subway, much of the direct responsibility for this was shouldered by another man, David Gunn, widely regarded as the most talented public transport operator in the world. Gunn was keen to come to London, just as Livingstone was keen to have him. But the dream ticket was scuppered by central government. Gunn, who can scarcely be accused of seeking an easy life since he now runs the US railway Amtrak, took one look at the Treasury's PPP arrangement for the running of the tube and decided the job was not for him after all.