Protesters attend a pro-mosque rally near the proposed mosque near the World Trade Center site on September 11, 2010 in New York City. Mario Tama/Getty Images(ALBANY, N.Y.) -- The backers of the controversial "Ground Zero Mosque" have won a court fight clearing the way for them to build the mosque and community center complex two blocks from the site of the 9/11 terror attack.

In a decision that was made public Wednesday, New York State Supreme Justice Paul Feinman dismissed a lawsuit by former firefighter Timothy Brown who argued that New York City was wrong to allow the destruction of a 150-year-old building to make way for the Islamic center.

The ex-firefighter who was among those who responded to the terror attack on the World Trade Center said the old building had been struck by debris during the collapse of the twin towers and was a "living representative of the heroic structures that commemorate the events of that day."

In a 15-page decision Feinman wrote, "Mr. Brown's claim that his ability to commemorate will be injured, is not yet recognized under the law as a concrete injury that can establish standing. Such an injury, although palpable to Brown, is immeasurable by a court."

The American Center for Law and Justice (ACLJ), a conservative legal group that filed the lawsuit on Brown's behalf, said they plan to appeal the ruling.

"This decision fails to give appropriate consideration to first responders and others who risked their lives and lost loved ones on Sept. 11," ACLJ attorney Brett Joshpe said in a statement.

The ACLJ "remain[s] confident that this mosque will never rise above Ground Zero."

Brown and the ACLJ were appealing a ruling last summer by the city's Landmarks Preservation Commission which decided to allow a 150-year-old Park Place building to be razed to make way for the center, a project dubbed Park 51.

The cultural center's chief organizer, Feisal Abdul Raufsaid he is "certainly in agreement with the ruling."

"Tim Brown is somebody who I met more than once and we expressed our sensitivities to the issues of the 9/11 families, many of whom are our friends. We have a strong commitment make sure whatever we do meets the sensitivity of the families," Rauf told ABCNews.com.

Rauf and other organizers, who include his wife Daisy Khan and Sharif el-Gamal, plan to erect a mosque and a $150 million "architecturally iconic" complex a couple blocks from Ground Zero to "benefit the whole community." But funding remains a question.

Rauf said that the organizers are putting together a capital campaign to fund Park 51. "A lot of things are being looked at, donors, loans and how to pay it off. There's a lot of work that goes into putting it together," he said.

Rauf said organizers don't expect to break ground for another three to five years.

Project 51 has been a source of national controversy since its unveiling last May. Opponents as well as supporters demonstrated at Ground Zero in reaction to the commission's decision to allow the mosque last August. Opponents were vocal in expressing their opinion that a mosque so close to the spot where Islamic terrorists destroyed the World Trade Center represented a victory monument to the attacks.

President Obama was drawn into the controversy when he initially endorsed the mosque. "As a citizen, and as president, I believe that Muslims have the same right to practice their religion as anyone else in this country," Obama said at a White House ceremony last summer that marked the Islamic holy month of Ramadan. "That includes the right to build a place of worship and a community center on private property in lower Manhattan in accordance with local laws and ordinances."

But in a visit to the Gulf Coast subsequent to that comment, Obama later dialed back his public support, saying that he supported the Muslims community's right to build the mosque, but was not sure it was a good idea to build so close to Ground Zero.