BRIDGEPORT -- Plans for 130 apartments in a cluster of decrepit brick factory buildings that started out with a Fairfield developer in 2008 are now in his mom's hands.

Garfield Spencer, who has been both criticized and praised for other major Bridgeport renovations, has made Ruby Spencer responsible for finally breaking ground, according to his office.

Multiple attempts to reach both were unsuccessful. Their longtime colleague, Aurora Leigh, project manager of the $25 million factory conversion on Cherry Street and Railroad Avenue, declined comment.

City planning officials are happy just to see some progress on the Spencers' five-year-old plan.

"You can see how hard it has been to make anything happen here," Bill Coleman, with Bridgeport's planning office, recently told the zoning commission of the site.

When the commission approved the factory reuse in mid-2008, just in time for the national economic collapse, the project was under Spencer's and wife Rebecca's First National Development company.

Their pitch?

"We will market the units to artists, craftspeople, telecommuters, first-time buyers, young couples, working singles and urban professionals" -- groups that city officials want living near downtown.

That approval was challenged in court by neighbor O&G Industries, a construction company worried the residential development was incompatible with the light industrial neighborhood.

Then last summer a spectacular fire that smoldered at the factory for days damaged one structure beyond repair.

While the economy was stabilizing, the Spencers reached a settlement with O&G preventing future apartment tenants from lodging noise complaints against the construction firm.

After the fire, the original plans were redrafted to demolish the fire-damaged portion and redistribute the 130 apartments among the four buildings left.

In late January the Spencers, using a new limited liability corporation in Ruby Spencer's name -- Cherry Street Holding -- renewed the 2008 zoning approval and are pursuing state housing funds and historic tax credits.

Bridgeport's planning office is eager for the Cherry Street/Railroad Avenue apartments to move forward because they will bring new life to an architectural eyesore visible from Interstate 95 and the train tracks.

David Kooris, hired last year as the city's planning and economic development chief, said that once they are built, the 130 units should attract plenty of takers.

"It's an opportunity for some pretty interesting space you're not going to get anywhere else in the region," he said.

The Spencers won accolades for similarly transforming the former Jefferson School and Warnaco factory into housing in the early and mid-2000s.

"Great project," Kooris said of the latter, dubbed Lofts on Lafayette.

Despite those successes, Garfield Spencer's reputation has taken some hits.

He was sued in 2009 by the Lofts on Lafayette Condominium Association and some of the individual unit owners over defective construction. The case was settled last September.

A month later, Spencer withdrew a countersuit filed against the attorneys who represented the condominium association-- Bender, Anderson & Barba P.C. of Hamden -- for defamation and intimidation.

In his complaint, Spencer cited an email the firm sent in 2009 to the Connecticut Housing Finance Authority, urging the state agency to reject the developer's future requests for financial assistance.

Although the Cherry Street/Railroad Avenue project is in Ruby Spencer's name, when the factory erupted in flames last June it was Garfield Spencer who made a dramatic entrance on scene. He was arrested for trying to approach the building and clenching a fist at a police officer who barred his way. Those charges were dropped.

The blaze, which according to the Bridgeport Fire Marshal's office remains an open case, is the second massive fire to strike one of Spencer's properties in two years.

In October 2010, flames destroyed an old mill in Pawtucket, R.I., that Spencer was converting to housing. A year earlier, Spencer was accused by Rhode Island authorities of allowing tenants to move into that site, the Union Wadding Mill, illegally before completion.

The Pawtucket blaze was ruled an arson by the Rhode Island fire marshal and remains an open case despite the offering of a $10,000 reward.

Asked if the unsolved Bridgeport factory fire will prevent the Spencers from moving forward with the renovations, police department spokesman William Kaempffer said: "If development plans progress and if a demolition permit was requested, the city would work with the developer and make that determination."