Those aforementioned characters you listed were established as the main antagonists/villains, though. Fujitora does not fit that role here; Doflamingo does. Fujitora seems to be what Kuma was to Thriller Bark or Kizaru to the Sabaody Archipelago. He's not the main threat that needs to be toppled, rather he's the more powerful adversary overseeing things and waiting to make a move.

Has Doflamingo really been established as the main antagonist (for Luffy)? Honestly Doflamingo is more Law's antagonist rather than Luffy's. Of every character actually on the proverbial battlefield Jesus or Tora are the more likely antagonist for Luffy purely due to their connection (however tenuous) to Ace (one is part of the crew that captured Ace, the other part of the group that killed Ace).