Why did evolution leave beards on men?

I've been googling this now and again for a while but found nothing at all that answers the question in an even half-satisfying way: You can't just say 'sexual selection' and leave it at that, else why did men not evolve to keep their tails who can say women wouldn't have fancied them more with a tail, or a furry ruff round the neck to keep warm whilst hunting..
We would all have had furry faces then at some point women's got bald and (many) men got left with the bottom half of their faces covered in hair. Why? Anyone got any ideas better than just 'sexual selection'? Cheers.

Modern humans have chins but the Neanderthals some of us interbred with didn't - so perhaps the beards are a relic of a time when only the Neanderthal males who could best disguise their chinlessness with beards were able to pass on their DNA, though this theory does not explain the puzzle of the chinless British aristocracy.

Modern humans have chins but the Neanderthals some of us interbred with didn't - so perhaps the beards are a relic of a time when only the Neanderthal males who could best disguise their chinlessness with beards were able to pass on their DNA, though this theory does not explain the puzzle of the chinless British aristocracy.

Modern humans have chins but the Neanderthals some of us interbred with didn't - so perhaps the beards are a relic of a time when only the Neanderthal males who could best disguise their chinlessness with beards were able to pass on their DNA, though this theory does not explain the puzzle of the chinless British aristocracy.

Modern humans have chins but the Neanderthals some of us interbred with didn't - so perhaps the beards are a relic of a time when only the Neanderthal males who could best disguise their chinlessness with beards were able to pass on their DNA, though this theory does not explain the puzzle of the chinless British aristocracy.

This is good, I like it.
Best I can come up with is something to do with it having been advantageous for men to be able to conceal their faces, maybe making them better at negotiations when meeting other competing social groups, whilst they liked being able to more easily tell what women were thinking.