Faust: damned if you do . . .

Goethe cursed attempts to set Faust to music – but composers kept trying regardless. As Terry Gilliam's version opens, Stuart Jeffries recounts a litany of depression, devils and duels

Orlin Anastassov as Mephistopheles in a 2006 production of Gounod's Faust at the Royal Opera House, London Photograph: Alastair Muir / Rex Features Alastair Muir / Rex Features/Alastair Muir / Rex Features

There is a curse on any composer rash enough to set Goethe's Faust to music. The German literary genius declared only Mozart capable of adapting his epic drama of damnation, sexual betrayal, witchcraft and freeform philosophic meditation. Selfishly, Mozart had died in 1791, almost 20 years before Goethe completed part one. So forever after, we have been doomed to suffer Faustian adaptations that the author would have disdained.

Perhaps Goethe's curse was issued because of That Thing he had with Beethoven. When Goethe met Beethoven (What a film! Hugh Bonneville as genteel, bewigged Goethe; Russell Crowe as Beethoven, surly and spoiling for a fight), the former bowed like a courtier; the latter didn't even remove his hat. You can see how that slight would get on your wick if you were Germany's greatest writer, how it would lead you to issue a teutonic fatwa against all new upstart composers to stop them mangling your meisterwerk.

Terry Gilliam, who will soon direct Berlioz's La Damnation de Faust for the English National Opera, knows more than most about the accursed task of adapting great literature. He once tried to make a film of Don Quixote with Jean Rochefort as Quixote and Johnny Depp as Sancho Panza, heedless of that sibling of the Goethe curse, the Cervantes jinx. Orson Welles began filming Don Quixote in 1957; when he died in 1985, it still wasn't finished. Similarly, when Gilliam tackled Quixote, Rochefort fell ill, the money ran out and instead of a film, we got Lost in La Mancha, a documentary about Gilliam's disastrous shoot (the rushes looked wonderful). I remember Gilliam shouting at me: "The curse of Don Quixote will never set me free!" But he still hopes to finish it and, you know, get closure.

While we wait to see what Gilliam does to Berlioz, let's consider what Goethe's curse has done to the many composers who have attempted his Faust. (We've ignored Fausts by Spohr, Busoni and Schnittke because, as you know, they weren't adapted from Goethe).

Berlioz's Huit Scènes de Faust (1829)

As a Parisian medical student, young Hector Berlioz read Nerval's French translation of Faust, saw himself as suffering the same mal de l'isolement as the protagonist and composed pieces depicting eight scenes. He sent them to Goethe saying he could not suppress a "crie d'admiration". Goethe's musical adviser Zelter condemned the works and the author never wrote back. Rude, really. Berlioz came to dislike the work and, having a much grander adaptation in mind, tried to destroy all copies of the piece. He failed – you can hear samples of Charles Dutoit conducting the Montreal SO in this intriguing curio here.

Conductor Thomas Beecham said that Berlioz's Damnation had "a bunch of the loveliest tunes in existence". The half-empty theatre that greeted Berlioz as he stepped on to the rostrum of the Opéra-Comique for the premiere in 1846 suggests Paris thought otherwise. The second performance was worse attended, the third cancelled. "Nothing in my career has wounded me more deeply," wrote Berlioz. With debts of at least 5,000 francs, he left for a nutty-sounding winter sulk-tour of Russia to rebuild his shattered fortunes. Berlioz freely adapted the play, even writing his own (rather wonderful) libretto for Faust's contemplation of nature (hear Jonas Kaufmann sing it superbly on YouTube), which Goethe would have thought a cheek.

Disparaged, as critic Hugh Canning pointed out,"for its stylistic inconsistency and alleged signs of incipient madness," this almost ludicrously ambitious work draws together elements of lied, grand opera, oratorio and church music to create what the Grove Dictionary of Music calls Schumann's magnum opus, and in so doing is truer to the source material than any other adaptation. Schumann takes on not just part one but part two, tackling Faust's later delusions upon hearing of a new world being created and its rapturous promise of an everlasting present. Watch Bryn Terfel sing Welch ein Morgenwölkchen Schwebet on YouTube. Then go out and buy the Britten-Fischer-Dieskau Aldeburgh 1972 recording and try to forget that only a few years after completing the opera Schumann jumped in the Rhine, was confined in a mental institution at his own request and died of tertiary syphilis.

Wagner's Faust Overture (1855)

Why, you may be asking, didn't Germany's leading opera composer get it on with the epic by Germany's most revered poet? Today, all we have is an overture, a fragment of a symphony. Listen to Bruno Walter conduct it on YouTube. In a fascinating lecture, Where Is Wagner's Faust?, Irmgard Wagner (no relation) argues the composer turned away from Faust because it "gave him the creeps". It spelled out too nakedly the protagonist's rejection of life, his death wish that leads him into betting with the devil. It's a theory. That said, Wagner's deep appreciation of Schopenhauer's renunciatory philosophy suggests it's not too plausible. But you never know.

Liszt had hated Goethe's hero. "Faust," he wrote, "seemed to me a decidedly bourgeois character. Faust's personality scatters and dissipates itself; he takes no action, lets himself be driven, hesitates, experiments, loses his way, considers, bargains, and is interested in his own little happiness." Yet after meeting George Eliot and George Henry Lewes and discussing the play, he composed this symphony in a white heat of inspiration, dedicating it to Berlioz. Listen to Bernstein conducting its final chorus and tell me there's anything bourgeois there.

Paris initially gave this Faust as rough a ride as Berlioz's. The National Opera House declined the opera, the Théatre-Lyrique delayed its appearance for a year and when it was staged the manager Léon Carvalho cut several numbers and (love this) insisted his wife sing the role of Marguerite. Now, though, Gounod's warhorse is one of the most performed of operas. It has an easy-going charm and is packed with memorable tunes. Wagner refused to see Gounod's opera, arguing his French friend had got the adaptation all wrong – an attitude shared by many fastidious opera critics since. Listen to Mirella Freni as Marguerite sing stunningly to Faust ("Go! You fill me with horror!") in the finale and wonder whether he was right.

Arrigo Boito's only completed opera premiered at La Scala, Milan. It provoked riots and duels over its supposed "Wagnerism" (How wonderful: wouldn't you like to live in an era when duels are fought over operatic interpretation? I would). He withdrew the opera from further performances to rework it, and it had a more successful second premiere in 1875. It is still frequently performed and recorded today. Listen to Pavarotti sing Giunto Sul Passo Estremo or Callas sing L'Altra Notte and then challenge anyone who doesn't rate them to a duel – possibly to the death.

The final section of the so-called Symphony of a Thousand is a setting of the closing section of Faust, part two – the depiction of an ideal of redemption through the Eternal Feminine, whatever that means. Critic Deryck Cooke argued that, like Beethoven's Ninth, "it sets before us an ideal [of redemption] which we are as yet far from realising – even perhaps moving away from – but which we can hardly abandon without perishing".

Eisler's Faust (1952, never completed)

Poor, adorable Hanns Eisler. He fled Nazi Germany after his music was banned, and settled in the US with his close collaborator Bertolt Brecht, only to be accused by McCarthyists of being the chief Soviet agent in Hollywood. Returning home to East Berlin, he hoped to write a modern opera on the Faust theme. The libretto, published in 1952, portrayed Faust as an indecisive person who betrayed the cause of the working class by not joining the German peasants' war, which, you'd have thought, would have gone down a treat. Not so: East Germany's national newspaper Neues Deutschland accused the work of being "a slap in the face of German national feeling", while Walter Ulbricht, general secretary of the Central Committee of the Socialist Unity Party of Germany (not a guy you would want to cross) denounced Eisler for having "formalistically deformed one of the greatest works of our German poet Goethe". Disheartened, Eisler gave up the opera. His remaining years were spent in depression and declining health. His music, though, endures – including a shattering setting of the Goethe fragment Von Wolken Streifenhaft Befangen.

The New York Times didn't like the idea of setting up Faust as an empty, disaffected Generation X kid: "He doesn't care about what happens to his soul. Why should we?" As a record, however, Faust showed genius; raspy-voiced Randy Newman played the Devil, but the masterstroke was having James Taylor play God as a vague, golf-playing bureaucrat out of touch with his constituents. Who wouldn't make a pact with the devil when God can be personified by someone who had a hit with You've Got a Friend?