Well first, there's the cop-out answer that there wouldn't be hate speech, or at least nowhere near as much, but the real answer is that it would have to be handled based on ties between individuals, like most things that are cultural in nature. If you faced ostracism for being a racist asshat, you wouldn't do it anymore, would you?