One of the worst thing is the AI of your companions. I was fighting some generic beasties and one of the companions was using a magical character to fight melee, despite having two melee characters in reserve. The worst part is that they had MP but weren't using that either.

Oh and I do find it very annoying that when one switches a character it automatically removes the familiar. It had been mentioned before on this thread and it's definitely a pain in the arse. As is the cooldown, sometimes it's a bit of a piss take.