(ABC News) – Rep. Alan Grayson, D-Fla., is defying criticism from Republicans and fellow Democrats for comparing the Tea Party to the KKK, saying he is “calling them out on their hate” and that it “needs to be done.”

Grayson ramped up his attack on the Tea Party after he was rebuked by Democratic Congressional Campaign Committee Chairman Steve Israel as well as from the GOP.

But Grayson was unapologetic.

“I’m calling them out for their hate,” Grayson said. “That’s not wrong. That needs to be done. It’s the only way to end it.”

He also went a step further Wednesday, attacking the Tea Party for being engaged in hate speech, prejudice, discrimination, racism and even warned that they could resort to violence.