Tax breaks for the wealthy are a contentious pre-election topic in the U.S., but this issue also strikes a chord with the Swiss voters. Although they pay lower taxes across all income brackets than Americans and other Europeans, many Swiss are increasingly critical of the preferential tax treatment their government extends to wealthy foreigners.

For decades, the rich who had been overtaxed in their own countries have been flocking to Switzerland to take advantage of fiscal perks most of its 26 cantons (equivalent of the U.S. states) are offering to well-heeled expats. But if a left-wing political alliance has its way, rich foreigners may soon have to start contributing a lot more money into their host country’s coffers.

Unlike Swiss citizens, who pay taxes not only on their income but also on their assets, wealthy foreigners can negotiate lump-sum taxation, a figure based on five times the rental value of their Swiss property. To qualify for the flat-rate tax deal, they are not allowed to work in the country but must have a net wealth of at least $2 million.

This system has attracted more than 5,000 affluent expatriates to this nation of only 8 million people, including singers Tina Turner, Phil Collins and Shania Twain; actress Sophia Loren; as well as Ikea’s founder Ingvar Kamprad. Although wealthy foreigners collectively shell out about $700 million in federal and local taxes, they save millions that they’d have to pay in their home countries.

But this arrangement may soon be a thing of the past. Taking advantage of Switzerland’s system of direct democracy, under which citizens can bring any issue up for a vote if they collect enough signatures on a petition, several organizations are now demanding the end of tax breaks for rich foreigners. “The lump-sum system undermines tax justice and goes against legal equality for all citizens,” Niklaus Scherr of the Alternative List, a socialist party and one of the groups behind the initiative, told a Swiss news service.

On Oct. 15, his group and other left-wing organizations submitted to the federal authorities a petition with 103,000 signatures — 3,000 more than required by law to launch a national vote. The government will schedule the referendum after a parliamentary discussion of this issue.

While many in Switzerland are calling for this change, there is also concern that elimination of tax perks will spark a mass exodus of affluent expatriates. “These people are a significant source of revenue and jobs in their communities,” says Stéphane Garelli, business and economics professor at University of Lausanne. “If we scrap their tax advantages, they’ll go spend their money elsewhere.”

In fact, at least one prominent foreigner is ready to pack his bags. Seven-time Formula One world champion Michael Schumacher, a German national who has been living near Geneva for the past 20 years, said he’d move out of Switzerland if the measure were to be approved by the voters.

Though he predicted that other rich foreigners would follow suit, so far nobody else threatened to jump ship. But even if they do leave and take their money out of the country, government finances would not suffer, Roland Meier, a spokesman for the Federal Department of Finance, tells TIME. He points out that the 2011 federal tax revenue — not including taxes collected separately by the cantons, which have fiscal autonomy to set their own rates — reached nearly $59 billion and the flat-rate taxation brought in “only” $200 million. Cantons and municipalities got additional $500 million in expats’ lump tax money.

Given those numbers, and considering that Switzerland has posted budget surpluses for several years in a row (with another one predicted for 2012), the loss of $200 million on a federal level would not make a big dent in its budget — though it might have more of an economic impact on cantons and municipalities that depend on the expat-tax revenue.

Schumacher’s departure, for example, would leave a substantial hole not only in the coffers of Gland, a small town of 11,000 residents where he lives, but would also wipe out the wages of employees who work at his sprawling $35 million waterfront estate.

However, the government is not turning a deaf ear to the critics of the current tax system. In September, the lower house of the parliament voted to maintain the lump-sum model while backing to raise the tax rate of rich expatriates to seven times the rental value of their home, from the current rate of five times. If passed by the upper house and unchallenged in a referendum, the new law would go into effect in five years. But the left-wing coalition that is pushing for the change argues that this measure is not enough and insists the flat-rate system be repealed altogether.

If the voters heed this call, life for the rich expatriates in Switzerland may get much more taxing.