Now is a good time to be running an oil company. Prices are sky-high, energy demand is increasing at an unprecedented rate as the global economy recovers, and there are new markets to be explored. Royal Dutch Shell largely dodged the criticism heaped on the industry after BP's catastrophic oil spill last year in the Gulf of Mexico, and is delivering golden results to shareholders. Profits for 2010 were reported as $18.6bn (about £11.4bn), nearly double the $9.8bn for the year before.

So it is not surprising that James Smith, Shell's outgoing UK chairman, speaks with a tinge of regret. At the end of the month, he will retire from the company he has been part of for 28 years, seven of them in his current role. Like other oil industry executives, Smith has a sense of destiny; an industry once regarded as dull and boring, a mere mechanical process of mineral extraction, has in the past five years taken centre stage in the global economy. "Energy is the most exciting industry," he says with enthusiasm, in a soft Scottish burr, in his soberly fitted 24th-floor office overlooking the London Eye and the Thames. "It represents the most invigorating challenge. Energy lifts people out of poverty, it enables development. It is a place where people can have excellent careers and great job satisfaction. I've always been excited to be part of it."

Smith, who has just turned 60, has overseen Shell's latest growth spurt, and its attempted transformation from a dirty industry into a cleaner, greener and more lucrative machine. Until recently, the company still had a link prominently displayed on its home page to its apology over the 1995 Brent Spar oil platform incident: Shell had intended to scuttle Brent Spar but it was eventually dismantled on land.

A physicist by training, Smith was born in Inverness. He qualified as a chartered accountant and while working at consultancy Accenture had a number of oil companies among his clients. The lure of the North Sea proved too much to resist and he joined Shell in 1983. Smith did his stint in a far-flung corner of the oil empire, as all ambitious Shell employees are required to do, spending four and a half years in Malaysia and Brunei along with spells in the Middle East and the US and as head of technology at Shell Chemicals.

Appointed chairman of Shell UK in 2004, Smith immediately recognised his tenure would be defined by his response to climate change. The European Union's emissions trading scheme – the first time that the oil and gas industry had to pay for its carbon emissions – came into force on 1 January 2005.

While oil industry chiefs in the US dismissed global warming as a fad, and tried to rubbish the science behind it, Smith was following in the rather greener footsteps of BP's Lord Browne.

There was more to it than copying his biggest rival, though, Smith protests. His education as a physicist had led to an early interest in the effect of greenhouse gases on the atmosphere. "I was worrying about climate change in 1973 when I was doing physics in Aberdeen. I have not stopped worrying since," he says. "Scientists and engineers do think about these things."

Smith saw that Shell would have to change and he led the group's programme to invest in renewable energy, building up stakes in plant-based biofuels and offshore wind energy. But these days those programmes – once held up by the company as evidence of its newfound caring, environmentally friendly image – are either defunct or largely irrelevant. Shell has pulled out of renewables: it retains a small stake in biofuels development, but the company's offshore wind business is no more. Shell announced in May 2008 that it was leaving the consortium building the world's biggest offshore wind farm, the London Array, and its interests in wind were in effect at an end.

Smith explains the demise of Shell's wind plans as a return to the company's core competences. "Wind is a manufacturing business, and we are not a manufacturer," he says. "You have to do what you're good at."

But there was another force at work. The oil industry had found a new green message, and one more palatable to its thirst for drilling and exploration: gas. The development of new techniques for fracturing shale rocks to release their load of natural gas has revolutionised the energy industry in the past three years. Suddenly, vast deposits of previously inaccessible gas have become available for exploitation.

The prospect of a global gas glut has sent prices plummeting, breaking the longstanding link between gas and oil prices, and has led to a bonanza among gas companies that, if fully realised, will dwarf the North Sea oil industry and lead to decades of cheap energy. As a bonus, that energy comes with about half of the carbon dioxide emissions from coal, enabling the fossil fuel to be labelled as "greener" than the alternatives.

Smith is fully aware of the potential. "Shell will be more of a gas company than an oil company within two years," he predicts, a huge change for a company with 50% more petrol stations than McDonald's has burger joints ("my proudest boast at Shell", he chuckles). It is hard to describe how big a change this represents to the energy industry; an abundance of cheap gas will transform its fortunes.

"If you can do carbon capture and storage with gas, then it can be a long term affordable source of green electricity," says Smith. "It gives the world a breathing space." What's more, the sources of shale gas are widely distributed around the world. "There might be 200 years of shale gas supply available," he notes.