At issue is occupancy of the new 42,000-square-foot building at 20 City Square. Neighbors say Doherty promised abutters that the three-story brick building would have retail space on the first floor and offices upstairs. Instead, 76 percent of the building has been leased to Homeland Security, which plans to house its offices and three holding rooms for detainees.

Nearly a week after a city agency declared that the Department of Homeland Security cannot move its Boston operation into a Charlestown office building, all sides in the controversy have dug in their heels for a long fight.

"We answered a request for proposal for first-class office space; I won that bid in a competitive process; the city issued a building permit; and I have signed a lease with a federal agency," said Timothy Doherty, project manager for Cornerstone Corp., the Norwood-based landlord. "I just can't walk away from the deal."

Neighbors say they will fight to make sure the deal is dead.

At issue is occupancy of the new 42,000-square-foot building at 20 City Square. Neighbors say Doherty promised abutters that the three-story brick building would have retail space on the first floor and offices upstairs.

Instead, 76 percent of the building has been leased to Homeland Security, which plans to house its offices and three holding rooms for detainees.

After a firestorm of protest by neighbors who envisioned gun-wielding law enforcement officers and suspected terrorists at the facility, Mayor Thomas M. Menino asked the Boston Redevelopment Authority to determine whether the holding cells meet the city's zoning guidelines.

Last week, the BRA wrote the General Services Administration, which handles real estate deals for federal agencies, saying detention rooms violate the city's zoning ordinances.

"As far as I'm concerned, Homeland Security should not move into that building in Charlestown," Menino said.

Despite double-digit office vacancy rates downtown, a GSA spokeswoman said her agency cannot consider any other location until the matter is settled between the landlord and the city.

"We've signed a lease that commits us to the City Square location, so we can't proceed contractually or financially with any other office building," said Paula Santangelo, a GSA spokeswoman. "But we won't give the landlord a green light to proceed with construction of the interior offices until the zoning questions are settled."

Susan Elsbree, the BRA spokeswoman, said the agency's interpretation of the zoning laws found that Cornerstone needed Zoning Board of Appeals approval to build holding rooms.

"We don't believe that holding cells are an accessory use to the office space given that it is not an allowed use anywhere else in the city," she said.

The matter could be settled if the Inspectional Services Department, the city's building permit granting agency, revokes the permit it issued to Cornerstone, Elsbree said.

But John Dorsey, an ISD spokesman, said he does not have grounds to revoke the permit.

"We found the holding rooms were consistent with zoning," Dorsey said. "Penal use is forbidden, but Homeland Security has said they will not be detaining anyone overnight or providing meals. Right now, we're waiting to see what GSA plans to do."

In the meantime, the neighbors are gearing up for a Zoning Board of Appeals or court fight.

Residents have raised more than $15,000 to hire a public relations firm and attorneys, according to Michael McCaffrey, who lives on nearby Main Street.