Mr Reckless, whose defection from the Tories to UKIP triggered the contest in Kent, travelled to London soon after his election to take his seat in Parliament, saying his constituents expected him "to get back to work".

As he was sworn in at the House of Commons, he was accompanied by UKIP's other MP Douglas Carswell, another former Conservative who won a by-election last month.

In other developments:

Shadow attorney general Emily Thornberry resigned from the Labour front bench just hours before the result after being accused of "sneering" by tweeting a photo of a Rochester house with flags and a white van outside

David Cameron said he was "absolutely determined" to win the seat back for the Conservatives in May's general election

Ed Miliband said the result was a "devastating" setback for David Cameron and he was determined to meet the challenge posed by UKIP

UKIP won its first seat on Medway Council after Conservative defector Chris Irvine was elected from the Peninsula ward.

Media playback is unsupported on your device

Media captionMark Reckless is sworn in as the second MP for the UK Independence Party

Image caption
The new UKIP MP has vowed to get back to business as soon as possible

Image caption
Mr Cameron said he was determined to win the seat back

Image caption
Labour has been hit by the resignation of its shadow attorney general

In his acceptance speech early on Friday morning, Mr Reckless said Rochester and Strood had been UKIP's 271st most winnable seat and "if UKIP can win here, we can win across the country".

Speaking to the BBC's political editor Nick Robinson, Mr Reckless urged other Conservative MPs to join UKIP but said it was difficult to predict how many, if any, would do so because it was a "very individual decision".

Election choice

The prime minister vowed to win the seat back in less than six months' time.

Media playback is unsupported on your device

Media captionLabour leader Ed Miliband says the by-election result is "a devastating result" for David Cameron

"This result was closer than the forecasters predicted," he said. "I am absolutely determined to win this seat back because absolutely anything other than a Conservative government will put our recovery at risk and mean Ed Miliband in Downing Street.

"I am more determined than ever that we deliver security for Britain."

UKIP leader Nigel Farage denied the result was simply a protest vote against the government, or that voters would return to the larger parties in the general election.

He said the message of Rochester and Strood was: "If you vote UKIP, you get UKIP" and "people out there who vote UKIP intend to stay UKIP".

He said he believed the number of seats that UKIP could win in May had "probably doubled".

Analysis

Image copyrightGetty Images

By Norman Smith, BBC assistant political editor

UKIP's victory was in many ways even more impressive than their triumph in Clacton. The ease with which they demolished a 9,000 Tory majority was striking and this after the Conservatives had strained every sinew to halt the UKIP bandwagon.

UKIP now insists no Tory seat is safe and has suggested other Conservative MPs are more likely to defect.

For the Tories the result was not perhaps the meltdown they had feared and certainly there is no indication so far of panic or calls for Mr Cameron to go.

Senior Tories also believe they are poised to win back this seat at the general election.

For Labour, not only was their share of the vote almost halved - but there was also despair at the damaging tweet by their former shadow attorney general Emily Thornberry, seemingly mocking "white van man".

As for the Lib Dems, not only were they again overhauled by the Greens, but they secured a derisory 349 votes - their lowest total ever.

Elections expert Prof John Curtice, from Strathclyde University, said UKIP still faced a challenge to convert its by-election success into seats at the general election.

He said: "It's certainly a very good night for UKIP, and makes life much more difficult for the major parties, but the challenge facing UKIP is clear."

Nick Robinson said it was "extremely unlikely" any further Tory MPs would defect in the short term, wary of having to fight a by-election so close to a general election.

Asked whether he could guarantee this would not happen, Chief Whip Michael Gove told the BBC's PM programme he was "convinced that nobody will".

Describing Mr Cameron as "the standout politician of our age", he said he believed he would lead the Conservatives to victory at the next election.

'Snobbery'

Labour was dealt a blow when shadow attorney general Emily Thornberry resigned from its front bench after being accused of "snobbery" by tweeting a photograph of a house in Strood displaying England flags and with a white van parked outside.

Media playback is unsupported on your device

Media captionThe moment Mark Reckless's victory was announced

The Islington South and Finsbury MP, who stood down after talks with Ed Miliband, said it had not been her intention to "upset or insult" anybody.

Shadow foreign secretary Douglas Alexander said Mr Miliband "did not hold back in making clear how angry he was" about the tweet.

He told the BBC that politicians could not take "any voter, community or class for granted" and must treat the public with a "deep and fundamental respect".

But Mr Farage suggested Labour was "increasingly anti-English".

"This Labour party or new Labour believes that any sense of English identity is disreputable and wrong," he said.