(CNN) – A well-known, openly gay supporter of Mitt Romney in New York has decided to withdraw his support for Romney and back President Barack Obama instead.

The clincher: Romney's stance on same-sex marriage.

"I feel that I no longer wish to support your presidential campaign and ask that you please return the maximum contribution that I gave to you last year," Bill White wrote in a letter addressed to the former Massachusetts governor and obtained by CNN.

"You have chosen to be on the wrong side of history and I do not support your run for president any longer," White added.

CNN contacted the Romney campaign but did not receive a response.

White is the chairman and CEO of the New York-based consulting firm Constellations Group. He was previously the president of the Intrepid Sea, Air & Space Museum – the cultural and educational institution whose major presence is the floating World War II Intrepid aircraft carrier sitting on the Hudson River. White told CNN he has advocated on behalf of injured and fallen veterans for 20 years - being awarded the Meritorious Public Service Award from the Coast Guard and from the Navy.

White told CNN he is a registered independent who has supported both Republicans and Democrats in the past, including former presidents Ronald Reagan, George H.W. Bush, Bill Clinton and George W. Bush. FEC records show that White contributed to both Republicans and Democrats over the years, including Hillary Clinton in 2008. Most recently, White gave $2,500 to Romney for President, Inc., according to FEC records.

White began his letter to Romney stating, "I am currently a max donor to your 2012 presidential campaign." He did not fundraise for the Romney campaign, though he said his partner and some friends also contributed to Romney.

In his letter, White goes on to mention the president's public announcement of his personal support for same-sex marriage last week.

"Several days later this past Saturday to a packed stadium of young college students you made the following statement that 'Marriage is only between one man and one woman,'" White wrote, referring to Romney's recent commencement address to Liberty University, an evangelical school in Lynchburg, Virginia.

"I believe that you will do as you now say and try to force a constitutional amendment which would attempt to make my own legal and blessed marriage null and void."

In an interview with CNN, White explained his stance.

"I felt we gave 'Hope and Change' a chance and I was looking for something different," White said.

"Quite frankly, I was not supporting Barack Obama – I was supporting Mitt Romney. And my support is not just words or my vote, it's also putting my money where my mouth is."

White said he does not agree with the president on fiscal policy – something he sides more with Romney on.

"I just think we're spending probably money that we don't have," he said.

And yet, White said: "I'm looking at the television screen one day and I see my president – whether I'm Republican or Democrat, he is my president…and I'm looking at him doing one thing, one day. And two days later, I'm looking at a guy who I'm supporting for president, and who I want to be my president and who I've given my money to, to be president, saying something I didn't think he would be saying."

CNN pressed White: Why now, given that Romney's stance is not new?

"I had a very visceral reaction to him – he had a great opportunity… to get on the right side of history," White responded. "And to be someone to the country that helps to unite us, versus what divides us. And I just think his very proactive intention to pounce on this issue now has sent me to the other side."

"Now, I feel like he's declared war on my marriage. And I could just sit back and not say anything. Or I could do something about it. And I've chosen to do something about it."

Following news of the president's decision, leading Republican LGBT advocacy groups expressed skepticism over its motivation. GOProud called Obama's announcement "hardly a profile in courage" and the Log Cabin Republicans labeled it "calculated." That suggests that the president's stance has not swayed everyone in the gay community to support him.

Meanwhile, Romney is not alone in seeing backlash from supporters.

The president has also faced problems over his stance. While some African-American pastors took to the pulpit Sunday to support the president, others were critical. One religious leader, Emmett Burns of Baltimore, supported Obama in 2008 but has now pulled his support.

"I love the president, but I cannot support what he has done," Burns said at the church.

Online Public Information File

Viewers with disabilities can get assistance accessing this station's FCC Public Inspection File by contacting the station with the information listed below. Questions or concerns relating to the accessibility of the FCC's online public file system should be directed to the FCC at 888-225-5322, 888-835-5322 (TTY), or fccinfo@fcc.gov.