Set back from the street among a lush green lawn and foliage, the Oakhurst Retreat and Conference Center, a 44-bedroom mansion in Northbridge, Mass., commands attention.

When the property came on the real estate market last May, Alain Beret and James Fairbanks of Sutton, Mass., jumped at the opportunity to purchase Oakhurst and its extensive grounds from the House of Affirmation Inc., an affiliate of the Worcester Diocese, as they were looking for an "exceptional property" where they could live and set up a new hospitality and special events business.

"We just fell in love with it," Beret told ABCNews.com, and the two men made an offer on Oakhurst. "We thought it had everything we were looking for to turn it into an inn, and we had said we intended to hold weddings, proms, various functions on the property." Beret and Fairbanks had made a business out of buying old properties, restoring them and turning them into special events venues.

But Beret and Fairbanks now find themselves in county court, waging a civil rights lawsuit against the House of Affirmation, the Roman Catholic Bishop of Worcester and others, claiming discrimination because of their sexual orientation, a violation of the Massachusetts General Laws. Beret, 59, and Fairbanks, 57, are married, and have been together for 34 years.

"Discriminating against a person who is gay is no different than discriminating against a person who is black, Latino, Jewish or female, said Sergio E. Carvajal, principal attorney at Carvajal & Nielsen, which joined with the Massachusetts Fair Housing Center in filing the suit in Worcester Superior Court.

Beret and Fairbanks, who are seeking unspecified compensatory and punitive damages, claim that in early 2012, according to the court filing, they began a search for "a unique and exceptional property where they could reside and establish a new hospitality and special events business. They had already owned and operated two other special events facilities, including one at the Harding Allen Estate in Barre, Mass.