U.K. Labour Party leader Jeremy Corbyn has called on Prime Minister David Cameron to consider taking direct control of Britain’s overseas territories in light of the Panama Papers leak exposing global tax avoidance.

Corbyn said that some territories, such as the Cayman Islands and the British Virgin Islands, were guilty of having lax laws that allowed wealthy individuals and companies to avoid tax.

The Panama Papers uncovered more than 100,000 secret firms were registered in the British Virgin Islands by the law firm at the heart of the controversy, Mossack Fonseca.

Overseas territories, which are self-governing and only rely on the U.K. for defense, can have their independence revoked. Most recently, the U.K. imposed three years of direct rule on the Caribbean Turks and Caicos Islands due to corruption among its rulers. Home rule was only reinstated in 2012.

“Surely to goodness we can at least stop tax evasion and avoidance in British controlled territories” — Jeremy Corbyn

Corbyn said that direct rule over other U.K. overseas territories could be instated “almost immediately” if the the government decided to do so and insisted it was “something that has to be considered.”

“Surely to goodness we can at least stop tax evasion and avoidance in British controlled territories,” he said. “The point is that they are not independent territories. They are self-governing, yes, but they are British crown dependent territories.”

“Therefore, surely, there has to be an observance of U.K. tax law in those places. If they have become a place for systematic evasion and short-changing the public in this country, then something has to be done about it. Either those governments comply or a next step has to be taken.”

However, Corbyn’s Conservative critics accused him of effectively attempting to resurrect the British Empire.

Conservative MP Graham Brady, who sits on the All-Party Parliamentary Group for the U.K.’s overseas territories, told POLITICO imposing direct control would backfire.

He said: “The problem of going back to a direct colonial relationship, quite apart from the fact that we are not supposed to believe in that sort of thing anymore, is that they would straight away declare independence. It would make the situation significantly worse.”

Former Conservative cabinet minister Dominic Grieve, speaking to the BBC, claimed any imposition of direct control to clamp down on tax avoidance risked bankrupting the islands.

He said: “If we’re going to destroy the economy of the British Virgin islands, which is because we prevent them from providing banking services at all, then we’re going to destroy the livelihood of its inhabitants.”

“I think the government has a responsibility towards encouraging overseas territories to find legitimate ways of economic development, and the financial sector is undoubtedly such a legitimate method.”

Former Labour Foreign Secretary Jack Straw also warned that direct rule was “easier said than done.”

But he said: “I certainly believe that given these revelations Jeremy Corbyn is quite correct to say the government has now got to come forward with a much tougher plan for the regulation of the British overseas territories.”