It turns out that a national controversy sparked by racist and homophobic remarks doesn't necessarily result in a ratings boost for a television show.

Many speculated that "Duck Dynasty," A&E's hit reality series about a family of swamp millionaires who made their fortune off duck calls, would draw even more viewers after one of the show's stars, Phil Robertson, made headlines last month for railing against "homosexual offenders," "Shintos" and "Islamists" in a wild interview with GQ.

But rather than a ratings spike, viewership for the show's season five premiere actually dipped considerably. Wednesday's premiere drew 8.5 million viewers, down 28 percent from the season four premiere last August.