Fox had no comment on the matter. A spokesperson for "The X Factor" told HuffPost TV the reports are merely speculation. "No one has discussed next years judging panel yet – any reports otherwise are complete speculation," the spokesperson said in an email statement.