The supreme court on Monday handed a victory to former President George W Bush’s attorney general, FBI chief and other officials, ruling they cannot be sued over the treatment of detainees, mainly Muslims, who were rounded up in New York after the September 11 attacks.

In a 4-2 decision with three justices not taking part, the court reversed a lower court decision that said the long-running suit could proceed against former attorney general John Ashcroft, former FBI director Robert Mueller and others. Mueller is now the special counsel investigating possible collusion between Russia and Donald Trump’s campaign team in the 2016 US presidential race.

The civil rights lawsuit sought to hold the former officials responsible for racial and religious profiling and abuse in detention that the plaintiffs said they endured after being swept up following the 2001 attacks by al-Qaida militants on the US.

Writing for the court, conservative Justice Anthony Kennedy said the allegations were serious and that the supreme court did not condone the treatment of the detainees. But, Kennedy said, the only issue for the court was whether Congress had authorized such lawsuits against public officials. It had not, the justices concluded.

“We are very disappointed with the court’s dismissal of our clients’ claims,” said Rachel Meeropol, senior staff attorney at the Center for Constitutional Rights, in an emailed statement.

“The court’s decision allows for high-level officials to violate the constitution without fear of personal accountability – a dangerous message in this time of rampant state-sponsored discrimination against Muslim and immigrant communities.”

The court sent one part of the case back to lower courts to determine if claims against Dennis Hasty, the warden in charge of the detention facility in Brooklyn where the detainees were held, could go forward.

The suit was filed by a group of Muslim, Arab and south Asian non-US citizens who, their lawyers said, were held as terrorism suspects based on their race, religion, ethnicity and immigration status and abused in detention before being deported.

The lawsuit claimed the senior Bush administration officials were liable because they made the policy decisions that led to the round-up and confinement of the plaintiffs.

Liberal justices Stephen Breyer and Ruth Bader Ginsburg dissented. Breyer took the relatively unusual step of reading his dissent from the bench, saying such lawsuits should be allowed to go forward in order “to provide appropriate compensation for those deprived of important constitutional rights and in times of special national-security need”.

The plaintiffs were charged with only civil immigration violations. But they said they were subjected at Brooklyn’s metropolitan detention center to 23-hours-a-day solitary confinement, strip searches, sleep deprivation, beatings and other abuses and denied the ability to practice their religion.

They said their rights under the US constitution to due process and equal protection under the law were violated.

During the US justice department’s massive investigation after the 9/11 attacks, certain immigrants in the country illegally were detained until being cleared of involvement.

The New York-based secnd US circuit court of appeals ruled that Ashcroft, Mueller and former Immigration and Naturalization Service commissioner James Ziglar could be sued, based on a 1971 supreme court precedent. In 2013, a judge dismissed the claims but allowed some against detention facility wardens.

Government lawyers have said there is no proof Ashcroft or Mueller personally condoned any potential unconstitutional actions.

The plaintiffs included Benamar Benatta, an Algerian Muslim who was seeking refugee status in Canada, and Ahmed Khalifa, an Egyptian Muslim who said he was on vacation.

In a similar previous case, the supreme court in 2009 backed Ashcroft, saying a lawsuit by detainees failed to contain specific details on Ashcroft’s involvement.