In 2001, nine of the top 10 most challenged books were on the banned book list because opponents said they had offensive language, sexually explicit passages or both.

The only one that didn't was Harry Potter. The series was challenged over its "religious viewpoint" — the only one of the top 10 to face that objection.

Fast forward to 2015, and five of the top 10 most challenged books faced opposition over their religious viewpoints. Librarians say it seems like people aren't taking issue with specific passages being inappropriate, but are against diverse themes like gender, sexuality and religion.

Reasons: Anti-family, offensive language, homosexuality, sex education, political viewpoint, religious viewpoint, unsuited for age group, and other (“wants to remove from collection to ward off complaints”).

Four of last year's most challenged books dealt heavily with LGBTQ issues. The third and fourth slots on the list were occupied by "I Am Jazz" and "Beyond Magenta," two books about transgender teens. In terms of challenges, they ranked just behind "Fifty Shades of Grey."

Alison Bechdel's "Fun Home" and David Levithan's "Two Boys Kissing" deal largely with gay teenagers and made it to numbers seven and 10 on the list, respectively.

James LaRue, who runs the American Library Association's Office for Intellectual Freedom, said the shift could be the waning white majority "lashing out to say, 'Can we just please make things the way that they used to be?'"

Though these books are challenged more than any other, the library association says it's usually able to keep them on library shelves.