If Libya's "Brother Leader" had flown to Europe for a G8 summit two
decades ago, his plane would probably have been shot down as a security
risk.

Yet last month, Col Muammar Gaddafi was among the selected leaders invited to the margins of the G8's conclave in Italy, where he was favoured with a meeting with Gordon Brown.

This week, the Libyan spy convicted for killing 270 people on board Pan Am Flight 103 may be released from Greenock prison in Scotland on compassionate grounds. If so, Abdul Basset Ali al-Megrahi will return to a Libya gripped by the transforming power of oil wealth and the end of international isolation.

The country he left to stand trial in 2001 was still an isolated and impoverished backwater, with sanctions having left Tripoli stuck in a time-warp of 1970s architecture and technology. Today, Col Gaddafi, now recast as a valued Western ally, is allowing BP to conduct its largest exploration project in Libya.

The British energy giant is scouring 21,000 square miles of the country's desert and coastline for untapped oil. Libya already possesses no less than 42 billion barrels of proven oil reserves – more than anywhere else in Africa - and many billions more probably lie undiscovered. On top of all this, Libya has at least 1.5 billion cubic meters of natural gas.

Nature has bestowed this largesse upon a country with only six million people, a rare combination which should, in principle, allow Col Gaddafi's domain to become one of the richest nations per capita in the world. But this hinges on Libya turning out as much oil as possible. Devoid of Western technology or investment, the state oil company produces only 1.8 million barrels a day - about half the amount that could be achieved. The arrival of BP and other Western companies should change this, allowing a projected doubling of oil output and placing tens of billions of dollars in Col Gaddafi's hands.

Other energy companies, notably from Russia, are trying to get in on the act. But Libya is unlikely to switch its favour, because only the Western giants, with the most advanced technology, can guarantee that production will reach its full potential.

The deal to allow BP back into Libya was struck in Col Gaddafi's unique fashion in 2007. The Brother Leader received Tony Blair in a threadbare, sparsely furnished tent, decorated with imprints of camels and pitched in a featureless expanse of Sahara near the Gulf of Sidra. Col Gaddafi, unshaven and swathed in brown robes, looked as unkempt and frayed at the edges as his tent.

Almost exactly 33 years earlier, this was the man who had burnished his credentials as a fervent socialist revolutionary by nationalising all BP's assets in his country. This time, the bemused figure of Peter Sutherland, the chairman of BP and a symbol of global capitalism, could be seen waiting outside the tent, mingling with an array of journalists and Libyan officials in Arab robes.

This may have been the first time that Mr Sutherland had been kept waiting beside a sand dune for an audience with a head of state, yet the agreement he and Mr Blair secured opened up vast new tracts for BP to explore. Before this meeting, Mr Blair disclosed that he was on first name terms with Col Gaddafi and paid tribute to the Libyan's reliability as an ally. "There is nothing I've ever agreed with him that should be done that hasn't been done," said the then Prime Minister.

How Col Gaddafi, who seized power almost exactly 40 years ago, went from being the "mad dog" of President Reagan's demonology, and the target of American air strikes in 1986, to a valued partner of Western capitalists and prime ministers is a remarkable diplomatic story.

Throughout the 1970s and 1980s, Col Gaddafi served as paymaster and arms dealer to an astonishing array of terrorist groups, including the IRA. A multitude of bloodstained African leaders enjoyed his favour, including Idi Amin of Uganda and Charles Taylor of Liberia, who is now standing trial for war crimes. From the 1990s onwards, however, Col Gaddafi one by one removed all the obstacles to better relations with the West. He cut his links with terrorists, accepted responsibility for the Lockerbie attack, paid compensation of almost £1.7 billion to the families of the victims - and handed over Megrahi for trial.

Then, in 2003, Col Gaddafi took his most significant step, disclosing his entire inventory of weapons of mass destruction to Britain and America, as well as opening to inspection all the facilities where they had been produced.

Oliver Miles, a former British Ambassador to Libya, told the Sunday Telegraph that Col Gaddafi came to recognise that "weapons of mass destruction were a blind alley. They were not giving Libya greater security, on the contrary they were putting Libya at greater risk".

During the Cold War, Col Gaddafi could be sure that America would not use force to topple him for fear of Moscow's reaction. But that assurance was removed by the Soviet Union's collapse and America's wars against Saddam Hussein in 1991 and 2003. "He saw that if Uncle Sam was going to give him a kick, there was no-one there to protect him," said Mr Miles.

Hence Col Gaddafi decided to bury the hatchet with the West. Mr Miles was the Ambassador who formally severed Britain's relations with Libya after WPC Yvonne Fletcher was shot dead outside the country's embassy in London in 1984.

Anglo-Libyan relations remained in the deep freeze until 1999, when the British Embassy in Tripoli was reopened after Col Gaddafi accepted responsibility for WPC Fletcher's murder and paid compensation to her family. He also promised to cooperate with a police investigation, although to date no-one has been brought to justice for her killing.

But was Col Gaddafi's thinking really dominated by oil? "They desperately need the Exxon Mobils of this world to sort them out, but I'm not really convinced that Gaddafi's moves were dictated by oil," said Mr Miles. "He thinks politically and strategically."

The obstacles to Libyan prosperity are now being removed, yet the Brother Leader's regime remains as corrupt and authoritarian as ever. There has been no hint of political reform, no talk of a Tripoli Spring to complement the new atmosphere of economic liberalism. "Libya's human rights record and continuing violations cast a shadow over its improved international diplomatic standing," according to Amnesty International's latest report.

Col Gaddafi has shown no willingness to repeal a repressive law, passed in 1972, which ensures that Libyans who "exercise their rights to freedom of expression and association may face the death penalty".

While Megrahi may be granted his personal liberty, he will not return to a free country. Nor will those who oppose his release on any grounds keep silent. Hillary Clinton, the secretary of state, has voiced America's objections, personally phoning Kenny MacAskill, the Scottish justice minister, to ask him to keep Megrahi behind bars. "We believe he should spend the rest of his time in jail," said the State Department.

If he is freed, however, this will mark yet another episode in Col Gaddafi's long rehabilitation.