Few films since 1988's Who Framed Roger Rabbit? have managed to collide the worlds of toons and humans without coming off as, well, a bit two-dimensional. I remember enjoying 1996's Space Jam as a kid, with its star pairing of Michael Jordan and Bugs Bunny; its theme tune, R Kelly's I Can Believe I Can Fly was the inescapable sound of school discos and embarrassing slow-dance boners. Even so, its visuals have aged as badly as a pair of Zubaz pants.

Ralph Bakshi's Cool World from 1992 doesn't fare any better for being pitched at grown-ups. Not even Brad Pitt's rockabilly detective can detract from the daft plot, in which a cartoonist (Gabriel Byrne) enters the unreality that his creations inhabit, and is seduced by a comic-strip Kim Basinger, who makes Jessica Rabbit look as racy as a sequinned thumb.

If anyone could master this surreal mash-up and bring it up to date, it should have been Israeli director Ari Folman. His 2008 animated documentary Waltz With Bashir was lauded for his distinct rotoscope style, similar to that of Richard Linklater's Waking Life and A Scanner Darkly. His latest film, The Congress, however, does nothing to convince us that the live action-animation format is anything but tricky.

Sorry, your browser is unable to play this video.

Robin Wright plays herself, an ageing Hollywood actress given one final contract. Her physical traits and emotions are captured by a Crystal Maze-like contraption, leaving the studio – the cheekily titled Miramount – free to use her image however they wish, and the former Princess Bride free to look after her ailing son. On the surface it's a takedown of the domineering studio structure wrapped up in a dystopian vision of cinema's future. Halfway through, however, as Robin returns to Miramount to renew her contract, she necks a vial of psychotropic potion and the world suddenly capsizes into an ocean ripped straight from the Beatles' Yellow Submarine.

Critics have called this topsy-turvy approach "ambitious", and it's true the film attempts to explore the many possibilities the dual formats offer. But it's as if the film-makers have licked too many acid tabs and forgotten to give their characters an exit door. It's hard to pinpoint the exact moment in The Congress that your own grip on reality starts to loosen. It could be when Michael Jackson serves up lobster to Wright and an animated animator (voiced by Jon Hamm). Or the bit where she's cryogenically frozen and wakes up in a time when everyone, like a bad Baby D lyric, lives in an ever-changing fantasy world in their minds.

Live action-animation films are often accused of not being "convincing", presumably because there is no conceivable universe in which toons and humans can both exist. At least Space Jam didn't alienate people more with a storyline that no one could follow, too. For everyone else, it's back to the drawing board…