NEW ROCHELLE, N.Y. (AP) — A “Don’t Tread on Me” flag hoisted by veterans outside a city-owned armory has been ordered down because of complaints that the defiant symbol of the American Revolution is now associated with the tea party and makes an unwelcome political statement.

Veterans in New Rochelle, just north of New York City, are threatening to sue, and a prominent conservative law firm has taken up the case. Some residents are upset with city officials, and a small town 60 miles away has raised the flag — with its coiled snake and yellow field — in solidarity.

“They shouldn’t be able to tell us what flag we can fly,” said Peter Parente, president of New Rochelle’s United Veterans Memorial and Patriotic Association. “They don’t tell us what color flowers to put at the veterans’ graves.”

But City Manager Charles Strome said: “They can fly whatever flag they want at the VFW post, at their own homes, but anything displayed on public property has to be determined by the municipality. Anything that could be offensive to some people ought not to be on public property.”

Both sides agree that the veterans group had permission to take down a tattered American flag at the vacant New Rochelle Armory and replace it with a new one.

Parente said someone brought the “Don’t Tread on Me” flag, also known as the Gadsden flag, to a sunset ceremony on March 21 and the veterans decided to fly it, given the flag’s historic military connections.

Several flags bearing a rattlesnake and the slogan were used during the Revolution, according to the Naval History and Heritage Command website. The rattlesnake was a common symbol for the United States and its resistance to tyranny, and the first commander in chief of the Continental Navy used the Gadsden flag in 1776, it says.

Since at least 2008, the flag has been used as an unofficial symbol of the tea party, the loosely defined populist movement that supports reduced government spending, lower taxes and reducing the national debt.

The flag is often seen flying at tea party rallies and at tax protests and gun-rights rallies. In 2011, Arizona authorized a tea party license plate featuring an image of it.

The flag’s tea party association has prompted complaints in recent years. At the Gettysburg Museum & Visitor Center bookstore, a shopper objected to the National Park Service selling Gadsden flag shot glasses, mugs, magnets and pins. In Folly Beach, S.C., the mayor got complaints because the flag was on a crane at a bridge-repair project. A Bethel, Conn., selectman said residents complained when a private citizen raised the Gadsden flag along with other historic banners on public property.

Parente said raising the flag at the New Rochelle Armory had nothing to do with the tea party.

“No one mentioned it,” he said. “I don’t think anyone knew. There is no organized tea party in New Rochelle.”

Strome said he wasn’t aware of the tea party link either, “but we got a few complaints that it was affiliated.” He said he did his own research and concluded, “They’ve basically taken the flag and written tea party all over it.”

Strome said he told the veterans it would have to be removed. But after they presented him with information about the flag’s long history, he said he agreed to hold off until he could check with city council members.

“The majority preferred to have the flag come down,” he said, and city workers removed it a week after it went up.

At an April 9 council meeting, members voted 5-2 against putting the flag back up. It was a party-line vote with a Democratic majority.

Councilman Jared Rice said the flag now symbolizes “a brand of politics that’s very offensive to a lot of people in America and a lot of people in New Rochelle,” which is more than 60 percent Democratic.

After the vote, several residents, one almost in tears, spoke in support of the veterans. A woman said her grandfather, father and uncle were all veterans, and she told council members, “You have disrespected them by taking down the flag.”

Parente said the Thomas More Law Center in Ann Arbor, Mich., contacted him and offered legal support.

The center’s president and chief counsel, Richard Thompson, said Friday that the city has violated due process, equal protection and free speech. Unless the flag is raised again at the armory, “I expect it to go to federal court,” he said.

Thompson said the veterans didn’t have specific permission to raise the Gadsden flag, but they never needed specific permission to raise the Stars and Stripes on city property, either.

“They had, in our mind, the authority to put up the Gadsden flag, which is an American flag,” he said. The Gadsden flag is “a part of our history that the tea party cannot take away. It’s quintessential America.”

Karl Brabenec, supervisor of the Town of Deerpark in Orange County, said he was at a gun rights rally when he heard about the New Rochelle dispute.

His town put up a donated “Don’t Tread on Me” flag on April 15 and “it will probably fly there forever,” he said.

The whole episode is “a slap in the face of veterans,” Brabenec said. “This is a historical flag. It represents freedom and sacrifice.”