An appointee of President Ronald Reagan, Kennedy has cast several deciding votes in cases related to abortion access, co-authoring the landmark 1992 opinion in Planned Parenthood v. Casey which upheld core tenets of Roe v. Wade and casting a swing vote in 2017 on Whole Woman’s Health v. Hellerstedt, which overturned Texas restrictions on abortion providers.

“With Justice Kennedy’s retirement, the future of Roe v. Wade and women’s right to choose is definitely at risk,” David Cole, the national legal director of the American Civil Liberties Union, told TIME in an email. “There are four votes on the Court now likely to be hostile to reproductive rights. All it takes is one more.”

In a statement, the anti-abortion Faith and Freedom Coalition essentially agreed.

The Brief Newsletter

Sign up to receive the top stories you need to know right now. View Sample

“President Trump won the evangelical vote overwhelmingly in 2016 in no small measure because he made an ironclad pledge that he would nominate pro-life, strict constructionists who would respect the Constitution, not legislate from the bench,” it read. “President Trump and the U.S. Senate have an ideal opportunity to restore the rule of law and create a culture of life.”

And the president of the anti-abortion Students for Life of America tweeted that the retirement would make it possible to “abolish abortion.”

THIS IS NOT A DRILL: Kennedy is RETIRING. We can overturn Roe and Doe. We can #abolishabortion in our lifetime. Get ready, #prolifegen! Lots of work to come! 2018 Elections will be key!

His appointment of Justice Neil Gorsuch in his first year in office was applauded by anti-abortion groups who are now pushing for him to appoint someone similar.

“President Trump now has another crucial opportunity to restore respect for life and the Constitution. We trust him to follow through on his promise,” said Susan B. Anthony List president Marjorie Dannenfelser.

Since Roe was handed down in 1973, a patchwork of laws have been introduced and passed across the states to regulate how and when women can access abortions.

Many of those cases have worked their way up to the Supreme Court — and as is often the case with social issues, they have been decided on thin margins. A shift of power on the court could eliminate the possibility that a swing vote could make a difference.

The issue of abortion is a live one at this moment. In 2017, 63 laws targeting abortion were enacted in 19 states, according to the Guttmacher Institute. There are also cases currently moving through the lower courts, including one conservatives are eager to argue before Supreme Court with hopes of overturning Roe.

In May, the ACLU and Planned Parenthood joined forces to challenge an Iowa law that bans abortions after six weeks, a time before many women even realize they are pregnant. One Republican state senator in Iowa told Reuters that with the law, they have created an opportunity “to take a run at Roe v. Wade — 100 percent.”

Still, Charles Geyh, an expert on the judiciary and professor at the Indiana University Maurer School of Law, said whomever Trump picks, the next court might still choose not to go at Roe v. Wade directly, instead choosing to chip away at the decision in a series of smaller cases.

“At a time when the Court’s legitimacy is jeopardized by the perception that the Court is, in effect, so many unaccountable politicians in robes the Court might approach Roe strategically,” he said, “so as to avoid the perception that a centerpiece of the Court’s jurisprudence is left to the vagaries of a single, very political appointment.”

But whether the court goes after court precedent forcefully or in smaller cases, abortion rights supporters like NARAL Vice President Adrienne Kimmell said they’re concerned for the future.

“Anti-choice extremists across the country have been very vocal, moreso than I have seen in my lifetime in this movement in saying that they’re expressed intent is to overturn Roe v. Wade,” she told TIME. “This is a pretty dire situation.”