(CBS/AP) BOSTON - A gay couple from Massachusetts is claiming that Roman Catholic Diocese of Worcester refused to sell them a former Catholic retreat center because church officials were concerned they would host gay weddings at the mansion.

James Fairbanks, 59, and Alain Beret, 57, filed suit in Worcester Superior Court for loss of civil rights and dignity and for emotional distress on Monday.

The married couple from Sutton planned to buy Oakhurst, a former Catholic retreat center in Northbridge, and restore it as a place they could live and host a special events business. Oakhurst was also previously used as a "House of Affirmation," a treatment center for priests with psychological problems, CBS station WBZ-TV in Boston reported.

But the Roman Catholic Diocese of Worcester ended negotiations in June, and the couple alleged they learned why in an email they inadvertently received in which diocesan Chancellor Thomas Sullivan cited concern "about the potentiality of gay marriages there."

"If you want from your pulpit to tell your congregants that you are against gay marriage, you have that right. But you don't have the right to chase me down the street with that," Beret told CBS radio station WBZ 1030 in Boston.

The couple's attorney, Sergio Carvajal, said it's obvious his clients were discriminated against because they're gay.

"It is wrong and it is illegal," he said.

But James G. Reardon Jr., an attorney for the diocese, said the diocese stopped negotiations over concerns about whether the buyers could finance the purchase. The email refers only to the possibility of gay weddings being held at the site, not the couple's sexual orientation, which Reardon said never came up during negotiations.

"It wasn't a case of discriminating against gay people. We didn't even know they were gay," Reardon said.

Reardon said the buyers' initial financing fell through, and they proposed carving out a 5-acre portion of the property for purchase. But that "made no economic sense to us," Reardon said, citing various associated costs, including surveying and plot-planning.

"That's why the deal fell through," he said.

The Oakhurst center is a 30-room mansion on 26 acres. It was built in 1890 by the descendant of a family that founded a prosperous local textile machine business.

According to the lawsuit, Fairbanks and Beret had converted another property into a successful special events facility and wanted to do it again.

The property was listed at $1.45 million, and the diocese accepted the couples' offer to buy it at $1 million, Carvajal said. But after an inspection revealed costly repairs were needed, they made a revised offer of $550,000. The diocese then ended negotiations in June.

The lawsuit charges that a June 8 email Sullivan sent to his real estate agent around that time proves the church discriminated against the couple.

"I just went down the hall and discussed it with the bishop," the email read. "Because of the potentiality of gay marriages there, something you shared with us yesterday, we aren't interested in going forward with these buyers. I think they're shaky anyway. So just tell them that we will not accept their revised plan and the Diocese is making new plans for the property. You find the language."

Carvajal said "the email speaks for itself," and that he doesn't buy that the diocese didn't know his clients were gay.

The diocese has a policy that forbids selling church buildings once used for worship for a purpose that's against church teachings, such as gay marriage. Carvajal said such a policy would be illegal in this case.

"They would basically be binding future people to discriminate on the basis of sexual orientation," he said.

Church officials declined comment on the lawsuit Wednesday, and Reardon did not comment on the policy.

Carvajal also denied that financing was ever a problem.

"At all times, my clients were qualified to buy this property," he said.