(CNN) - The Super Bowl champion Pittsburgh Steelers made the traditional winner's trip to the White House Thursday - but one of the team's star players was conspicuously missing from the lineup.

Linebacker James Harrison told reporters earlier this week he'd be a no-show at the White House. He denied any political motivation for his decision. "This is how I feel - if you want to see the Pittsburgh Steelers, invite us when we don't win the Super Bowl. As far as I'm concerned, he would've invited Arizona if they had won," said Harrison, who later joked that he was staying away because the White House was located in a "bad neighborhood."

This isn't the first time Harrison has declined an invitation to visit 1600 Pennsylvania Avenue - he turned down an invite from former President Bush after the Steelers won Super Bowl XL. "Let me ask you a question," he said Wednesday, according to the Pittsburgh Post-Gazette. "Why is it a big issue now that I'm not going if it wasn't a big issue the last time?

"...Hey, James ain't changed. I guess my profile did, but I didn't change. I'm not going because I don't want to go," he said. "They're making a big deal out of this: 'Oh, my, James Harrison is not going to the White House; he must be a devil worshipper!'"

Teammates have said the linebacker - whose 100-yard touchdown with an interception made Super Bowl history this year - is uncomfortable flying.

Harrison's boss, Steelers owner Dan Rooney, was a major supporter of President Obama's 2008 campaign, and was recently appointed ambassador to Ireland.The president openly rooted for a Steelers victory this year.