It's never acceptable to judge a man by the color of his skin -- but sometimes it's wise to judge a man by the logo on his t-shirt.

Idaho police say a neo-Nazi picked a fight with an African American man and got knocked out with a single punch. Not surprising, considering the man who threw the punch was wearing a shirt from a Spokane, Wa. boxing club.

Daren Abbey, 28, threatened to stab 46-year-old Marlon Baker in a Bayview, Idaho bar on July 3 and told him "blacks are not welcome here," Kootenai County Sheriff's Office sources told NWCN.

Baker and his friend left the bar, with Abbey allegedly following behind shouting racial slurs. Then Abbey made "a motion like he was going to do something," Baker told KREM.

Baker threw a single punch, knocking Abbey to the ground, investigators said.

"[It was] just a straight right hand, that's all it was," said Baker. "It's really good that he stepped into it."

Abbey was treated for a possible broken nose. He is being held on $75,000 bond on charges of battery and malicious harassment.

Countering witness reports, Abbey insists he's the victim and that Baker attacked him from behind with a "rock or some kind of object."

"I would not just go around threatening people or saying I was going to stab somebody just because they were not white," he told the station.