David Akin, National Bureau Chief

Flaherty -- one of the longest serving finance ministers Canada has ever had and the only one to ever serve Prime Minister Stephen Harper -- quit Harper's cabinet Tuesday.

His replacement, to be named Wednesday morning in a ceremony at Rideau Hall that will be closed to the media, appears to be Oliver. Sources tell QMI Agency Oliver is the man for the job and several other media outlets are also reporting the same thing. Aides in Harper's office were not responding to messages Tuesday night.

Oliver, though, was seen as one of the likely contenders to take Finance the minute news of Flaherty's resignation broke shortly after stock markets closed Tuesday afternoon.

Oliver is a bilingual 73-year-old Toronto MP who spent years on Bay Street as an investment banker.

He understands and speaks the world of money markets and central bankers and, from a political standpoint, is unlikely to aspire to succeed Harper one day.

Oliver will be responsible for designing the 2015 budget, one in which a surplus of several billions will mean Oliver's toughest job will be to decided whether to shower Canadians with tax cuts or new spending -- all of it in an election year.

Jim Flaherty tries on new shoes at the Roots Factory in Toronto prior to the federal budget announcement in this combination of file photos from March 20, 2013. (Stan Behal/QMI Agency)

Flaherty, unluckily perhaps, recorded just two surplus budgets in the 10 he delivered since 2006. His most famous budgets were the first two when he cut two points off the hated GST.

Oliver had been Harper's natural resources minister. It's not yet clear who will take that key post as the point person on the Keystone and Northern Gateway pipelines and other energy issues.

Flaherty had told reporters several times in the last several months that he was staying to fight the 2015 general election but, in fact, he and his family had decided months ago to let go of the job and look to return to the private sector.

Flaherty has spent two decades in politics in Ottawa and at Queen's Park. His aides said he plans to stay on as the MP for Whitby-Oshawa "for the time being."

Thank you @JimFlaherty - the best Finance Minister in the world - for your years of dedicated service to Canadians. #cdnpoli

Flaherty has suffered over the last year from a rare and painful but treatable skin disorder. In his statement, Flaherty said his health did not play a part in his decision to quit politics.

In a statement, Harper thanked Flaherty for his service.

"It is with great reluctance that today I accepted the resignation from cabinet of Jim Flaherty," Harper said.

Opposition Leader Thomas Mulcair, who shares Irish ancestry with Flaherty, also thanked him for his service.

In his resignation statement, Flaherty said that he was most proud to be part of a government that lowered corporate taxes to the lowest level in the G7, introduced the tax-free savings account and reduced taxes 160 times.

But Flaherty will also be remembered for bringing in the single largest deficit in Canadian history, a response to the financial crisis of 2008-09. Of the ten budgets Flaherty tabled since 2006, all but two were deficit budgets.

In the 2013 budget he tabled this spring, Flaherty predicted the federal government would be back in balance next year.