Further, 23% of the show's own viewers said Fox should scrap the show, while 58% said Fox should keep it on the air.

That's a notable shift from last week, when 41% of Americans said the network should scrub The O'Reilly Factor and 28% favored keeping the show on the air. The shift came as more people became aware of the scandal involving O'Reilly — 50% of survey respondents this week said they'd heard of the allegations against O'Reilly, compared with 44% who said they'd heard about them last week.

The New York Times reported earlier this month that five former coworkers alleged repeated instances of verbal abuse, lewd comments, and unwanted advances that resulted in settlements paid by Fox News and O'Reilly himself.

Through a representative, the Fox News host has denied the allegations, saying he settled complaints quietly not out of guilt but because he felt the accusations would harm his family.

O'Reilly's future at the network has become the subject of speculation since the Times report was released.

On Tuesday, New York magazine reporter Gabriel Sherman reported that the network was leaning toward pulling O'Reilly from the air permanently. The Wall Street Journal reported that Fox was preparing to cut ties with O'Reilly as early as this week.

Before O'Reilly left for a prescheduled vacation last week, numerous prominent advertisers dropped ads from "The O'Reilly Factor." Protesters have gathered outside the network's headquarters on multiple occasions calling for Fox News to dismiss O'Reilly.

O'Reilly has been the reigning king of prime-time cable news for years.

Amid record highs across the cable landscape in the first quarter of 2017, "The O'Reilly Factor" garnered its highest ratings in the two decades it has been on the air and broke the record for highest viewership for a television program in a quarter. The day after the Times report, 3.65 million viewers tuned in to the show, over a million more viewers than the preceding and following Fox News programs.

A rotating cast of Fox News personalities have filled in for the embattled host while he is gone, but none have replicated his ratings.