James Fairbanks and Alain Beret saw potential in the property – 44 rooms, 26 acres and community zeal for preservation. The stately though deteriorated mansion in Northbridge, Mass., seemed an ideal location for an inn or a special events.

But despite an offer and a deposit, the seller – the Worcester Diocese of the Catholic Church – backed out of the deal.

The diocese says the deal-breaker was money. The couple claims it was discrimination.

The vacant property at the center of the controversy most recently hosted the Oakhurst Retreat and Conference Center, an office for the church’s youth ministry.

Before that, beginning in 1973, the mansion was the site of the House of Affirmation, a retreat for “troubled priests” founded by the Rev. Thomas A. Kane. In a notice in a church-affiliated newsletter from decades ago, Kane announced that the center was at “the service of all priests and religious who are not embarrassed to become a more fulfilled and healthier person.”

Treatment, he wrote, could help a troubled priest become an “affirmed person,” leaving behind “neurosis, emotional and mental discomfort, alcoholism and addiction, erratic homosexuality, compulsive heterosexual behavior, and other symptoms of unhappiness and confusion.”

Over the years, with increased attention and disclosures about child sexual abuse in the priesthood, “troubled priests” would be revealed to mean “pedophile priests.” Priests accused of abusing kids were sent to the House of Affirmation in Northbridge for treatment or to hide out.

The House of Affirmation closed in the late 1980s, as Kane was accused of financial improprieties and falsifying a doctoral degree. He was placed on leave after at least one allegation of sexual abuse, which was settled out of court for a church payment of $42,500 to the victim, who said Kane abused him for 11 years, including at the retreat.

The Bishop Accountability Project, which tracks priests accused of sexual abuse, says several other men have settled suits against priests associated with the retreat. One victim, who received a $110,000 settlement from the diocese in 2002, recently called the House a “dirty, dirty place” to the Worcester Telegram & Gazette. Others have described the facility as a boot camp for pedophiles.

In the Worcester area, the property is associated with the scandalous, not the sacred.

But the property’s history begins before the House of Affirmation, and it is a history that the citizens of Northbridge have shown an interest in preserving.

The Chester Whitin Lasell family, which made its fortune in the textile-machine industry, built the mansion in 1890. The property is the last of its kind in the area and voters, at a special town meeting in February, elected to zone it into a historic district.

Beret called the mansion a “grand dame” – though she’s in need of more than cosmetic surgery.

The couple reduced their offer from $1 million to $550,000 after learning the extent of repairs needed.

“We expected that we would continue our dance, but the dance partner left the room,” Beret said.

The husbands, in a discrimination suit filed on Sept. 10 in Worcester Superior Court, claim the diocese didn’t sell because they are gay and, it was assumed, would hold gay weddings on the premises.

Church officials are only commenting through their attorney, who maintains that the seller was concerned about Fairbanks and Beret coming through with the financing. “It wasn’t a case of discriminating against gay people. We didn’t even know they were gay,” attorney James G. Reardon Jr. told The Associated Press.

He said the couple wanted only to buy a fraction of the property, which didn’t make sense to the church.

However, an email from the diocese dated June 8 – around the time the real estate bargaining ended – lends support to the couple’s claim.

“I just went down the hall and discussed it with the bishop,” Diocese Chancellor Thomas Sullivan allegedly wrote to his real estate agent. “Because of the potentiality of gay marriages there, something you shared with us yesterday, we aren’t interested in going forward with these buyers. I think they’re shaky anyway. So just tell them that we will not accept their revised plan and the Diocese is making new plans for the property. You find the language.”

Fairbanks and Beret, who are suing for loss of civil rights and dignity and for emotional distress, said that the subject of gay weddings never came up during discussions about their purchasing the property,

“There was never, ever a discussion about gay marriage,” Beret said.

The attorney for the couple, Sergio Carvajal, said regardless of whether the business plan included hosting gay weddings, the church broke a state law that prohibits discrimination in housing.

“It was a facility we were extremely interested in,” Fairbanks said. “We have made our life by restoring old buildings.”