A good compromise leaves everyone angry. And so it is with NFIB v Sebelius, the Busby Berkeley-scaled conclusion to this year's supreme court term, in which John Roberts sided with his four liberal benchmates to save President Obama's most important domestic achievement. The government cannot order you to buy health insurance, Roberts ruled – but it can tax you if you don't, and that's what the Affordable Care Act does.

Good enough. But not very satisfying.

The chief justice found common cause with the court's liberal bloc for the first time since his ascent in 2005, and as soon as his decision came down (or at least, after CNN and Fox News stopped reporting that he'd ruled the other way) movement conservatives started comparing their former superman to David Souter, that Republican turncoat. Really, though, his decision reminds me of something by Sandra Day O'Connor – whom Roberts was originally meant to succeed before his sudden nomination for the top job. The decision is pragmatic, conservative, and so minimalist as to strain credulity.

Roberts' opinion opens with a lofty hymn to limited government, going all the way back to the drafting of the bill of rights, and positions the individual mandate as an unprecedented dilemma. Does the constitution permit the federal government to require Americans to buy health insurance? Not under the commerce clause, Roberts says, since it does not regulate commerce but rather "compels individuals to become active in commerce." He goes on:

"Construing the commerce clause to permit Congress to regulate individuals precisely because they are doing nothing would open a new and potentially vast domain to congressional authority. Every day individuals do not do an infinite number of things."

That last line sounds a lot like something Marcel Duchamp would say, but no matter. For Roberts, the main distinction is between activity and inactivity. This supposedly radical incursion into individual Americans' freedom – which did not seem so radical when the Republicans proposed it as an alternative to "Hillarycare", or when Mitt Romney enacted a nearly identical program in Massachusetts – could open the door to anything:

"Indeed, the government's logic would justify a mandatory purchase to solve almost any problem. To consider a different example in the healthcare market, many Americans do not eat a balanced diet. That group makes up a larger percentage of the total population than those without health insurance. The failure of that group to have a healthy diet increases healthcare costs, to a greater extent than the failure of the uninsured to purchase insurance. Those increased costs are borne in part by other Americans who must pay more, just as the uninsured shift costs to the insured. Congress addressed the insurance problem by ordering everyone to buy insurance. Under the government's theory, Congress could address the diet problem by ordering everyone to buy vegetables."

Recognize that last line? It's the noxious "broccoli" argument, a Tea Party cock-and-bull story elevated to law by the chief justice of the United States. But as the unflagging Ruth Bader Ginsburg explains, in her often hilarious separate opinion, "although an individual might buy a car or a crown of broccoli one day, there is no certainty she will ever do so;" nor will she get broccoli for free "at the expense of another consumer forced to pay an inflated price". Ginsburg calls this freshman slippery-slope reasoning "the broccoli horrible", and she mocks her conservative benchmates for imagining that "a vegetable-purchase mandate" could bring down the healthcare costs of "lithe Americans":

"The court would have to believe that individuals forced to buy vegetables would then eat them (instead of throwing or giving them away), would prepare the vegetables in a healthy way (steamed or raw, not deep-fried), would cut back on unhealthy foods, and would not allow other factors (such as lack of exercise or little sleep) to trump the improved diet."

Unlike broccoli, healthcare is something everyone, but everyone, will need at some time. For Ginsburg and the other liberal justices, the individual mandate is not the unprecedented dilemma Roberts insists it is. There is nothing particularly new here. On the contrary, it fits in naturally with what Ginsburg calls "Congress' large authority to set the nation's course in the economic and social welfare realm", and failing to recognize the legislature's power to do so under the commerce clause is for her "stunningly retrogressive".

A few liberals have got themselves worked up that Roberts' decision is a Trojan horse, allowing Obamacare to stand while limiting federal power. But if this is what a federalist victory looks like, I'd hate to see defeat. Since he and four others found the mandate proper under the taxing and spending clause, the chief's writing on the limits of the commerce clause writing is just dicta (the legal term for the nonbinding parts of an opinion): a future ruling may take them into account, but it may not. In any event, even if this decision did lead lower courts to limit the applicability of the commerce clause, Congress can now just dress up new regulations in tax clothing as opposed to interstate commerce garb. So Roberts has struck what is, at best, a symbolic blow.

The more consequential part of the ruling, in fact, may be the less visible one: its limitations to the expansion of Medicaid and, by extension, on congressional spending power. Remember that unlike Medicare, Medicaid is run by the states. But it's done along federal guidelines, and Washington stumps up most of the cash. The ACA would have provided states with what Elena Kagan described at oral arguments as "a boatload of federal money" so they could offer Medicaid to all Americans with incomes up to 33% above the poverty line – about $31,000 a year for a family of four. But the federal government would withdraw existing subsidies from states that didn't go along with the expansion.

Not cool, the court ruled: states can stay with the Medicaid they've got now and not be penalized for it. The carrot remains, but the stick is gone.

This is bad news, for health reform and for jurisprudence. The limitation to congressional spending power, which basically says that the federal government cannot coerce states into obedience, could call all sorts of popular federal programs into question – think of Title IX, which transformed women's athletics in America, or the Clean Air and Water Acts.

And as for the 17 million working-class people who should have benefitted from Medicaid expansion, the fate of their healthcare is up in the air. It's easy to say that the whiny Republican state governors who want to resist the federal government's supposed intrusion into their own affairs will come around. The Medicaid expansion is actually a jackpot deal for the states, especially poorer southern ones; besides, these states' citizens have already paid up to the IRS, so it'd wouldn't be very nice to deny them.

But who knows? These are the same governors who turn down free money for shiny high-speed rail or impose "job-creating" 0% income taxes; and millions of people may both fail to get Medicaid coverage and still be ineligible to buy on an insurance exchange.

How could the president and the Democratic party have avoided this mess? There was a way: a public option, which passed the House, which had majority support in the Senate, and which would have been the law of the land had Joe Lieberman not denied the Democrats a supermajority. A public option would have been cheaper, fairer, more popular, and – like Medicare or social security – prima facie constitutional.

But this being America, that probably wouldn't have stopped someone from suing.

• Quotations from Chief Justice John Roberts' written opinion are given without interpolated citations; for the full original is available from the US supreme court site (pdf)