As Liam Scarlett premieres his first full-length narrative ballet, it's clear he has some very nasty things in the woodshed of his imagination. Not even the brothers Grimm could rival the troubling nightmare into which Scarlett has transformed the story of Hansel and Gretel.

The witch in the woods is dead, but in her place is a paedophile, obsessed with children and toys, and a Sandman accomplice, who comes in the grotesque form of a giant ventriloquist's doll. As these two lure Hansel and Gretel into the basement of their cottage, they open up a hellish fantasy world peopled with the spirits of Dr Coppelius, Norman Bates and Josef Fritzl.

It's a fantasy that Scarlett has updated, Matthew Bourne-style, to 1950s America. Courtesy of Jon Bausor's excellent designs, we're shown the peeling-lino poverty into which the children's family has slipped – a claustrophobia of unpaid bills and broken dreams that parallels the scary basement on the opposite side of the stage. There seems little chance of a happy ending in so confined a world, and the decision to have the audience seated along two sides of the stage, coupled with the small space of the Linbury, gives added visceral discomfort to that confinement.

The scale of the venue does, however, work against Scarlett as well as for him. Often the choreography feels squeezed for space; and while Leanne Cope and James Hay give alert, detailed performances as the two children, their first playfully innocent scenes are marred by overprojected body language and facial expressions. It's as though we're in The Nutcracker, not Psycho.

But if these two need to scale back their acting, Bennet Gartside and Laura Morera as their parents evoke a wonderfully credible dynamic of mutual disappointment and diminishing sexual attraction. Even better is Steven McRae's Sandman, his mercurial dancing filtered through a hellish lurch and floppiness of gait, and Brian Maloney as a pale, twitchy monster of fetishised desires.

This ballet is very nearly a dark, brilliant success, but what prevents it from being so is the occasional flatness of Dan Jones's score and Scarlett's own pacing. His choreography focuses too much on the children's struggle with their captors, not enough on the darkening shifts in their personalities, while the creepy denouement feels rushed. Despite its flaws, however, this Hansel and Gretel makes me impatient to know what other stories Scarlett has lurking in the woodshed.

• What have you been to see lately? Tell us about it on Twitter using #GdnReview