Now that the Supreme Court has taken up a challenge to the health care law, advocacy groups on both sides of the ideological spectrum are hoping to get a justice –with potentially opposing views– dismissed from hearing the challenge.

The focus for now from conservative groups is Justice Elena Kagan, while left-leaning groups turn their attention to Justice Clarence Thomas.

The absence of one justice on a close case could change the ultimate outcome. But at this point the possibility of either Thomas or Kagan stepping off the case, with the information that is publicly available, is remote. Recusal issues are largely left up to the individual Justice and if one had decided not to participate in the closed door deliberations regarding the case it would have been reflected in the order list released from the court last Monday.

“It’s pretty clear they have both decided to sit,” said Monroe Freedman, a Professor at Hofstra Law School.