Washington (CNN) -- The Army rescinded its invitation to evangelist Franklin Graham for the upcoming National Day of Prayer at the Pentagon over controversial remarks he made about Islam.

"True Islam cannot be practiced in this country," he told CNN's Campbell Brown last December. "You can't beat your wife. You cannot murder your children if you think they've committed adultery or something like that, which they do practice in these other countries."

Graham later tried to temper his remarks by saying that he had Muslim friends. However, the son of famed evangelist Billy Graham has a history of comments that bothered the Pentagon. Shortly after the terrorist attacks of September 2001, for instance, Graham called Islam a "very evil and wicked religion."

Graham, who said a prayer at the inauguration of President George W. Bush, said he regretted the Army's decision but stood by his comments.

"I don't like the way they treat women, the way they treat minorities. I just find it horrific. But I love the people of Islam," he said, adding some of his work has been in Muslim nations. For instance, Samaritan's Purse, the international charity that he heads, works with Iraqi refugees in Jordan.

"It's a part of the world I love very much," Graham said. "And I understand it. But I certainly disagree with their teaching."

"I want to express my strong support for the United States military and all our troops," he said. "I'm very proud of them and it's just unfortunate that I won't be able to participate on May 6th."

The Army, which oversees the National Day of Prayer ceremonies at the Pentagon, feared that if Graham spoke at the Pentagon, Islamic militants would publicize his comments, potentially fueling tensions in Muslim nations like Iraq and Afghanistan, where U.S. troops are deployed.

Concerns about Graham were flagged by the watchdog group Military Religious Freedom Foundation, which also raised objections that the Pentagon prayer ceremony had become a fundamentalist Christian event.

The group penned a letter to Defense Secretary Robert Gates on behalf of Muslim military members and defense department employees. It said that Franklin's remarks had sparked outrage.

"Mr. Graham has never retracted or apologized for these statements," the letter said.

But Graham's supporters disagreed with the Army's decision.

"What are they afraid Franklin Graham is going to say?" said Rep. Randy Forbes of Virginia. "I think again we are getting to the point in the country where we are trying to exclude everybody from speaking if I disagree with what they are going to say."

Graham's invitation was not the only controversy swirling about the National Day of Prayer this year. Last week, a federal judge struck down as unconstitutional the 1952 law that established the day, saying it violated the ban on government-backed religion.

On Thursday, the Justice Department informed a federal appeals court that the Obama administration will appeal that decision.