In most places fairies morph changing from dark to light and good to
evil quickly. This is why I call fairies internally duelistic because
they are their own opposites. In ancient teams people believed that
fairies were astral beings of the freest form, able to change shape,
form, and even personality at will.
When you feel the presence on a dark night that chills you to the bone.
When a soft moan carries through the house or lonely moor, do not fear
the ghosts, fear instead the fairies. For it is fairies which ancient
peoples believed haunted the darkness.
In mythology fairies turn dark when they witness a murder or suicide.
Thus many of the myths about ghosts and the origin of ghost stories
comes from tales of fairies. Of course many fairies were believed to be
the spirits of the dead, but haunting activity usually occurred
when tree and rock spirits were angered.

It wasn't building on graveyards that people feared it was building on
the rocks which fairies lived in, it was hurting plants they considered
sacred.