Justice Antonin Scalia’s angry dissent in the case over Arizona’s harsh immigration law is filled to the brim with partisan bile and dismissive rhetoric directed at his colleagues on the court. Scalia directly criticizes Obama’s immigration policy of deferring deportation for potential DREAM Act beneficiaries and describes Arizonans as being “under siege by large numbers of illegal immigrants who invade their property, strain their social services, and even place their lives in jeopardy.” Scalia also suggests that “we should cease referring to [Arizona] as a sovereign State.”

But even among these eyebrow-raising passage is one section that stood out from the rest. Explaining why he would have let the Arizona law stand in its entirety, rather than invalidating most of its provisions as the court majority ultimately did, Scalia runs through some of the history of immigration law in the US and cites slavery-era statutes meant to restrict the movement of free blacks across state lines.

Notwithstanding “[t]he myth of an era of unrestricted immigration” in the first 100 years of the Republic, the States enacted numerous laws restricting the immigration of certain classes of aliens, including convicted crimi­nals, indigents, persons with contagious diseases, and (in Southern States) freed blacks. State laws not only pro­vided for the removal of unwanted immigrants but also imposed penalties on unlawfully present aliens and those who aided their immigration.

While this might seem like an odd way to support his argument, Adam Winkler, a professor at the University of California-Los Angeles School of Law, warns against reading too much into it. “I think what he’s getting at is that there was a time when states had authority over this issue,” Winkler says, “but that was when Congress hadn’t enacted significant regulation on immigration.”

With the exception of one provision, which Justice Anthony Kennedy’s opinion implied could be invalidated in the future, the high court agreed that the states don’t get to legislate where the federal government has already set immigration policy. Or as Winkler appropriately puts it, given the Obama administration’s record of aggressive enforcement of immigration law, “the federal government is the only one authorized to make life miserable for undocumented immigrants.”