Sanford, a Republican who admitted an extramarital affair in 2009, was ready to quit politics for good if he was not victorious in South Carolina's 1st Congressional District. He will replace Republican Tim Scott, who was appointed to the Senate.

The former governor - once a rising GOP star considered presidential material - was an early favorite in the Republican district, which Mitt Romney carried by 18 percentage points in the 2012 election. But the revelation that Sanford's ex-wife, Jenny, accused the candidate of trespassing at her home caused the National Republican Congressional Committee to withdraw its support in mid-April and gave Colbert Busch an opening.

At his victory party, Sanford thanked the "angels" he met during the campaign and spoke of learning about "human grace" as reflected through a higher power. "I just want to acknowledge a God not just of second chances, but of third, fourth, fifth, sixth, seventh and eighth chances ... because that's the reality of our shared humanity," he said.

Sanford must appear in Charleston County Family Court on Thursday on his ex-wife's complaint. It is not clear when he will be sworn in to office.

Colbert Busch, a businesswoman and sister of comedian Stephen Colbert, stayed competitive but a final Public Policy Polling survey two days before the special election showed a dead heat. A business development officer who worked in the shipping industry, Colbert Busch focused on her plans to create jobs and vowed she would be an independent voice for the district.

"The people have spoken and I respect their decision," she told supporters Tuesday night.

This was the first time Sanford was on the ballot since his admission in 2009 that he was hiking on the Appalachian Trail - when he was actually in Argentina visiting his mistress - became a punch line on late-night TV.

"I absolutely failed. Period," Sanford said in an interview earlier Tuesday on CNN, one of several he gave on Election Day. He told MSNBC that "one event does not define your life."

Sanford urged his friends and neighbors to help him spread the word about his ideas to curb the nation's debt and to send a message about his Democratic rival. He stressed that Colbert Busch received help from labor unions and liberal groups from outside the district, and tied her to House Democratic leader Nancy Pelosi.

Rep. Greg Walden of Oregon, chairman of the House GOP campaign committee, congratulated Sanford and made no mention of how the organization parted ways with the candidate. "These results demonstrate just how devastating the policies of Barack Obama and Nancy Pelosi are for House Democrats in 2014," Walden said in a statement. "At the end of the day, running on the Obama-Pelosi ticket was just too toxic for Elizabeth Colbert Busch."

Rob Godfrey, a spokesman for South Carolina Gov. Nikki Haley, said the governor congratulates the congressman-elect. Haley, who appeared at a fundraiser for Sanford last week, "looks forward to working with him to bring jobs to the Lowcountry, fight job-killing unions, rein in out-of-control Washington spending, turn back Obamacare and strengthen the Charleston port," Godfrey said.

Democrats, meanwhile, saw a silver lining in Colbert Busch's efforts.

Rep. Steve Israel of New York, chairman of the Democratic Congressional Campaign Committee, said the candidate's efforts will be an example of how the party will stay aggressive in the 2014 elections. Democrats need a net gain of 17 seats next year to win majority control in the House. Sanford's election does not change that equation.

"The fact that the Democrat made this competitive is a testament to the strength of Elizabeth Colbert Busch as a candidate and the Republican habit of nominating flawed candidates," Israel said. "Democrats will be aggressive and drive deep into Republican-held territory this cycle to find districts with flawed candidates where we can compete."

The House Majority PAC, which spent $450,000 on TV ads and direct mail to boost Colbert Busch, said the outcome in South Carolina reflects poorly on the GOP.

"The House Republican caucus has added yet another ethically challenged embarrassment who will be an albatross around the neck of every Republican forced to answer for Mark Sanford's embarrassing and reckless behavior," said Alixandria Lapp, executive director of House Majority PAC.