Japan's main opposition party has chosen a nationalist leader who is tipped by some to become prime minister by the year's end, raising fears of an increase in tensions with China and South Korea over rival territorial claims.

Shinzo Abe, a former prime minister who resigned in 2007, citing ill health, won a run-off against his nearest rival, Shigeru Ishiba, to become leader of the Liberal Democratic party (LDP) on Wednesday afternoon.

"I'm still responsible for causing all of you trouble with my sudden resignation as prime minister six years ago," he told party members. "I will do my utmost with you to regain power."

Some analysts believe Abe, 59, could lead the LDP to victory in an election that could be called as early as November.

The party was ousted by the Democratic party of Japan (DPJ) in 2009, ending more than 50 years of almost unbroken conservative rule. The leader of the biggest single party in the lower house of parliament is practically assured of the prime ministership.

The LDP is expected to capitalise on the unpopularity of the current prime minister, Yoshihiko Noda, who is facing calls for an early election in the wake of controversial tax reforms and his handling of the economy. Noda's position has been further weakened by party defections and popular anger at his decision to restart two nuclear reactors this summer.

He has also been criticised for not taking a tougher stance against China during the recent flare-up in the countries' long-running dispute over sovereignty of the Senkaku islands, known as the Diaoyu in China.

The prospect of Abe as prime minister is likely to cause disquiet in Beijing and Seoul at a time of mounting tension over rival claims to island territories. Aside from its row with China, Japan is in dispute with South Korea over two islands in the Japan Sea, referred to as the East Sea by Koreans.

During the recent party leadership campaign, Abe emerged as the most hawkish candidate on territorial and historical disputes with Japan's neighbours.

"Japan's beautiful seas and its territory are under threat, and young people are having trouble finding hope in the future amid an economic slump," he said. "I promise to protect Japan's land and sea, and the lives of the Japanese people, no matter what."

He has said he wants to water-down a 1995 statement by the then socialist prime minister, Tomiichi Murayama, apologising for Japan's wartime aggression, and to withdraw a 1993 apology for its use of Korean women as sex slaves before and during the war.

During his time as prime minister, Abe called for the revision of Japan's pacifist constitution and for patriotism to become part of the school curriculum.

"I think China will be alarmed, as will Korea," Koichi Nakano, a professor of political science at Sophia University, told Reuters. Nakano, however, said Abe's victory on Wednesday could benefit the DPJ government.

Opinion polls indicate the LDP would not win an outright majority in a general election, and could turn to members of a new party formed by the rightwing populist mayor of Osaka, Toru Hashimoto.