But leader Nick Clegg hailed the win as "stunning" and "against the odds", given the Lib Dems' poor showings in recent national opinion polls.

UKIP candidate Diane James came second with 11,571 votes, her 28% the party's best-ever showing in a Westminster election.

Conservative Maria Hutchings won 10,559 votes, representing a 14 percentage points reduction the 2010 general election, while Labour's John O'Farrell came fourth with 4,088 votes.

Speaking in Downing Street, Mr Cameron said: "It is a disappointing result for the Conservative Party, but it is clear that, in mid-term by-elections, people want to register a protest.

"But I am confident that at the general election we can win those people back by demonstrating that we are delivering for everyone who wants to work hard and wants to get on. That is what we will be focused on."

Analysis: Reasons for voting

By David CowlingEditor, BBC Political Research Unit

An "on-the-day poll" conducted among people who had voted in the Eastleigh by-election suggests immigration was the biggest single issue for people supporting UKIP. Some 55% mentioned it, compared with 15% of all voters.

As for Europe, 9% of all voters mentioned it, compared with 31% of UKIP voters.

Meanwhile, 85% of Lib Dem voters said they had the best candidate, compared with 77% of Conservatives and 46% of UKIP voters.

And 83% of UKIP voters said they wanted to send a message they were unhappy with the party they usually support nationally, compared with 35% of all voters.

Mr Cameron insisted the Conservatives would not move to the right to encourage voters back from UKIP, saying: "I don't think we should tack this way, tack that way."

He promised to continue cutting the deficit, helping people "who work hard", reforming welfare and cutting immigration.

Mr Cameron said: "That is my agenda. That is their agenda. This is a by-election. It's mid-term. It's a protest. That's what happens in by-elections.

"It's disappointing for the Conservative Party but we must remain true to our principles, true to our course, and that way we can win people back."

But some Conservative MPs raised questions about the leadership.

Party vice-chairman Michael Fabricant wrote on Twitter: "The Conservative voice is muffled and not crisp. It does not clearly project Conservative core policies or principles."

He added: "UKIP appealed to protest voters but also to Blue Collar Conservative voters."

Backbencher Eleanor Laing said people felt "hurt" by the way they were being treated, telling BBC Radio 4's The World At One: "Ordinary Conservative voters don't feel that this government is in tune with them, with their hopes and fears.

"The only way to take forward those issues that people really care about is to have a truly Conservative government. And to do that the leadership of my party has to tune in better to the people who want to support it - who want loyalty and who now feel rather left out."

Living standards

Another Conservative MP, Douglas Carswell, said there were "policy lessons" to be learned and questioned why the party was on a "long march of defeat".

He told the BBC: "I would like us to do far more to focus on the bread-and-butter issue of cost of living. I think living standards are declining.

"We talk about only a protest vote. One of the reasons why people feel inclined to protest is because they are hurting in their pocket."

But Sarah Newton, deputy chairman of the Conservative Party, told BBC News: "Actually, it's good for the coalition that the coalition has kept a seat despite all the difficulties facing the Liberal Democrats and the coalition."

Conservative MEP Daniel Hannan said the Tories and UKIP should start "having conversations" and "stop calling each other names".

But UKIP leader Nigel Farage continued to be critical of the prime minister.

He said: "The Conservatives failed here because traditional Tory voters look at Cameron and they ask themselves 'Is he a Conservative?' and they conclude 'No, he's not.'

"He's talking about gay marriage, wind turbines, unlimited immigration from India. He wants Turkey to join the European Union.

"The Conservatives' problems are not because of UKIP; they are because of their leader."

Lib Dem victor Mr Thornton, who has been a parish and borough councillor since 2007, said: "The people of Eastleigh recognise that the Liberal Democrats have always had a superb record of delivery, we've always listened to what people want, and we always make sure that we do a good job."

Labour leader Ed Miliband said he would have preferred to have done better but it was "tough" territory for his party because it had never come close to winning Eastleigh before, even in its 1997 landslide election year.

The result, he added, showed Labour needed to "redouble its efforts" to increase its appeal to voters, in the south of England and elsewhere, who were not traditional supporters.