Vampires, fallen angels and their brooding kin still crowd the young-adult shelves of your local bookstore. But they are having to make room for a new wave of dystopian fiction, kicked off by the jaw-dropping success of Suzanne Collins's The Hunger Games trilogy, set in a post-apocalyptic North American totalitarian state.

Books for young people set in a post-apocalyptic or dystopian worlds are not new. Three notable early examples are Madeleine L'Engle's science fantasy A Wrinkle in Time (1962), William Sleator's suspense novel House of Stairs (1974) and the politically intriguing The Giver (1993) by Lois Lowry. Some of the big names of the new wave, along with Collins, are British-based American author Patrick Ness, Mortal Engines writer Philip Reeve, and young adult science-fiction novelist Scott Westerfeld. But what is it that attracts teenage readers to dystopian fiction?

There are a number of opinions, but the main drift seems to be that books set in either chaotic or strictly controlled societies mirror a teenager's life; at school, at home, with their peers and in the wider world. Let's call it the "my own private dystopia" theory.

I'm going to offer a much simpler explanation. Teenagers like to read dystopian fiction because it's exciting. It all comes down to the story. The story comes first, and the setting – extraordinary though it may be – is of secondary importance.

For the most part, dystopian fiction owes more to myth and fairytale than science fiction. These are essentially heroes' journeys – they just happen to be set in an imagined future world. The hero, reluctant or willing, is just as likely to be female as male. Something happens – an event, or a messenger arrives bearing news – and the teenage protagonist is catapulted out of their normal existence into the unknown. They cross the threshold into a world of darkness and danger, of allies and enemies, and begin a journey towards their own destiny that will change their world. They will be tested, often to the very edge of death. The stakes are high. The adults are the oppressors. The children are the liberators. It's heady stuff, far removed from the routine of everyday life.

The outer, global journey of the characters is matched by an inner, emotional and psychological journey. These are no cartoon superheroes. They, like their teen readers, have to deal with recognisable concerns and problems, including friendship, family, betrayal, loss, love, death and sexual awakening.

A new wave of dystopian fiction at this particular time shouldn't come as a surprise to anyone. It's the zeitgeist. Adults write books for teenagers. So anxious adults – worried about the planet, the degradation of civil society and the bitter inheritance we're leaving for the young – write dystopian books.

We create harsh, violent worlds. These are dark, sometimes bleak stories, but that doesn't mean they are hopeless. Those of us who write for young people are reluctant to leave our readers without hope. It wouldn't be right. We always leave a candle burning in the darkness.