A federal judge has handed the Washington Redskins their greatest loss yet in a more than 20-year legal battle over the team’s trademarks.

U.S. District Judge Gerald Bruce Lee ruled Wednesday in favor of five Native Americans who argued that the Redskins trademark is derogatory and ordered the cancellation of the team’s federal trademark registrations. The judge found that the six trademarks at issue “consisted of matter that ‘may disparage’ a substantial composite of Native Americans.”

The ruling does not prevent the team from using the contested designs and logos but would eliminate some of the legal protections that come along with federal registration of a trademark.

The trademarks cancellations won’t take effect until the National Football League team has exhausted all of its options in the federal appeals process, which would head next to the U.S. Court of Appeals for the 4th Circuit and possibly the U.S. Supreme Court.

Redskins President Bruce Allen said Wednesday the team was surprised by the ruling and will appeal the case.

“We are convinced that we will win because the facts and the law are on the side of our franchise that has proudly used the name Redskins for more than 80 years,” Mr. Allen said.

Judge Lee’s ruling upholds a decision made last year by the U.S. Patent Office’s Trademark Trial and Appeal Board, which called the trademarks “disparaging” and ordered a cancellation of the registrations.

The Redskins sought to overturn the ruling by filing a lawsuit in federal court in Alexandria, Virginia, against the five Native American activists who pursued the trademark cancellation. The team argued that a cancellation of the trademarks would violate its First Amendment rights to free speech.

“The federal trademark program is government speech and is therefore exempt from First Amendment scrutiny,” Judge Lee wrote in his 70-page opinion.

As part of their case, the Native American activists had to prove that the term “redskins” was offensive at the time the contested designs and logos were trademarked — forcing them to examine the use of the word spanning back to 1967. They cited dictionary definitions, scholarly articles, newspaper clips, and statements made by Native American groups and individuals at the time to prove their case.

An attorney representing the five Native American activists said they were gratified to have won this step of the legal battle.

“This decision is a victory for human dignity and for my courageous clients who have waited so long for this ruling,” said attorney Jesse Witten.

Even if the team were to lose a future appeal, the loss of the federal trademark is more a symbolic blow than a fatal one. Legal experts have said the Redskins‘ trademarks would still be protected under common law, and the team would be able to pursue legal action against anyone who used the logo or name without permission, for instance to make knockoff hats and jerseys.

But the case marks the team’s biggest loss in a trademark battle waged by activists since 1992.

The Trademark Trial and Appeal Board previously stripped the Redskins of trademark protections in 1999 based on a similar claim filed in 1992. However a federal judge overturned the board’s decision in 2003 on the legal grounds that the plaintiffs waited too long to bring their case — 25 years passed between 1967, when the first trademark was granted, and the time the case was brought.

The group of plaintiffs involved in the latest case are younger than those involved in the first case.

The ruling in the initial case did not render a decision on whether the name Redskins was offensive, rather it simply spoke to the Native American group’s lack of evidence and the “legal sufficiency” of the trademark board’s decision.