The easy answer of course is that libertarians are hateful people. But when you translate "libertarian" from American into English you get "classical liberal" which means us. And we're lovely, cuddly, people so that cannot be the right answer. Over in the US they're trying to answer this question (Tyler Cowen, Bryan Caplan) and there's a variety of reasons given. None of which quite explain it all to our satisfaction.
So, we'll put forward two more, the first not entirely serious. Which is that we're right, they know we're right, we know we're right and everyone hates smug gits who know they're right and who everyone knows are right.

The second, entirely serious, reason is that we're the only people not telling people how to live their lives. On the right, the conservatives, want to insist that everyone keep it in their pants until married, don't ingest things that make you feel good, work hard and if you conform to our prescriptions then maybe we'll let you be. On the left we've the usual hodge podge but the same urge is there. Live your life according to the manner in which we demand you live your life. Don't be greedy, don't be too successful or we'll take it all off you, you must respect everyone's decisions about how they live their lives and don't, whatever you do, blame anyone for creating their own bed that they've now got to sleep in.

By contrast we're the only people saying that we don't give a damn how you live your life. As long as you're not harming others, nor their ability to live their life as they wish, why on earth should we even pay attention to how you live let alone control, or even respect, how you do?

Which is why they all shout at us of course. For if you're running around with a set of rules that everyone must live by it's OK to have another group running around with a different set of rules about how you must live. But it's disconcerting, discombobulating even, to have a group insisting that there is no such list so would everyone shut up please? What merit in gaining control of the State in order to force everyone to follow your prescriptive rules if the libertarians (or classical liberals) have got there first and removed the power of the State to insist upon everyone having to follow any set of such rules?

To put it very simply, those who fight to insist upon vanilla flavoured cake are quite happy to battle those who fight for chocolate flavoured cake and vice versa, but they'll unite in hatred against those who say the cake is a lie.