iStock/Thinkstock(WASHINGTON) -- The U.S. Supreme Court sided with the state of Ohio Monday – upholding its practice of purging people from registration rolls if they fail to vote.

In a 5-4 decision, the court ruled Ohio could continue to remove individuals from voter rolls if they had not voted in two federal elections and have not responded to a confirmation notice or updated their registration.

The plaintiffs in the case, Husted v A. Philip Randolph Institute, had argued the Ohio law violated the National Voter Registration Act – that "just as you have a right to vote, you have a right not to vote," claiming the state's purges violate federal law and risk disenfranchising eligible voters.

The initial lawsuit — filed by the ACLU of Ohio and the public policy organization Demos – against the state's top elected official, Secretary of State Jon Husted, alleged voters will be "denied the opportunity to cast a vote that counts."

Ohio's practice of deeming voters ineligible affects tens of thousands of voters in the state — many from low-income communities who might move more frequently than more affluent voters. The ACLU cites a Reuters analysis that found in Ohio’s five largest counties, residents of predominantly African-American neighborhoods were about twice as likely to be purged as were residents of predominantly white neighborhoods.

Ohio claims its practices just keep voter rolls up to date and are consistent with federal law.

Justice Samuel Alito wrote the majority opinion, with two dissents: one from Justice Stephen Breyer, for himself and the three other liberal justices, and one from Justice Sonia Sotomayor.

"This case presents a question of statutory interpretation, not a question of policy," Alito wrote. "We have no authority to second-guess Congress or to decide whether Ohio’s Supplemental Process is the ideal method for keeping its voting rolls up to date. The only question before us is whether it violates federal law. It does not."

Sotomayor's dissent reads in part: "Communities that are disproportionately affected by unnecessarily harsh registration laws should not tolerate efforts to marginalize their influence in the political process, nor should allies who recognize blatant unfairness stand idly by."

"Today’s decision forces these communities and their allies to be even more proactive and vigilant in holding their States accountable and working to dismantle the obstacles they face in exercising the fundamental right to vote," she wrote.

Among the comments in the suit, a Butler County voter, Chad McCullough found out in 2016 that he was no longer registered to vote when he headed to the polls to cast his ballot.

"I had no idea that my voter registration could be canceled, even if I did not move," McCullough told ABC News in 2016.

In his comments, McCullough said, he "usually only vote[s] when there is an issue or candidate on the ballot that [he] care[s] about."

During oral arguments in January, Ohio State Solicitor Eric Murphy reiterated to the court that individuals who didn’t vote in two election cycles were also sent a notice that had to be signed and returned.

"So long as we send individuals a notice and so long as we wait two federal elections before we remove them, that that is acceptable," Murphy said.

Responding to the court's decision Monday, Sen. Sherrod Brown, D-Ohio, chastised the decision, writing in a statement, "Today’s decision empowers Ohio to further strip away the right to vote for thousands of Ohioans, threatening the integrity of our state’s election process."

"This ruling further shows why we can’t afford to pack our federal courts with judges who have a track record of hostility towards Ohioans’ most basic right," he added.

The ACLU called the decision "a setback for voting rights" and "a stark reminder that the Trump administration wants to turn back the clock on voting rights."

Demos, which led the legal team on behalf of the A. Philip Randolph Institute, said in a statement that the "decision threatens the ability of voters to have their voices heard in our elections."

"The fight does not stop here," Stuart Naifeh, senior counsel at Demos. "If states take today’s decision as a sign that they can be even more reckless and kick eligible voters off the rolls, we will fight back in the courts, the legislatures, and with our community partners across the country."

Husted lauded the court's ruling, calling it a "victory for election integrity."

"Today’s decision is a victory for election integrity, and a defeat for those who use the federal court system to make election law across the country," he said in a statement. "This decision is validation of Ohio’s efforts to clean up the voter rolls and now with the blessing nation’s highest court, it can serve as a model for other states to use."