Friday 25 April 2003 21.15 EDT
First published on Friday 25 April 2003 21.15 EDT

In 1855 Johannes Brahms wrote the pianist Clara Schumann a naked cry of frustration: "I can do nothing but think of you... What have you done to me? Can't you remove the spell you have cast over me?" The situation between them at the time was messy - very messy. Clara was 35, Brahms 21, she famous, he rather more infamous. She was married to the composer Robert Schumann, and the pair had seven young children. On the other hand, for more than a year, Clara's husband had been in an asylum and Clara had not been allowed to see him. When Robert fell off the edge, Brahms had hastened to her side.

Now Brahms, Robert's protege and discovery, was helplessly in love with Robert's wife. They had not expected it, didn't want it, and so on. Brahms loved and admired Robert. Shortly before jumping in the Rhine to escape the demonic oratorios in his head, Robert had made the name Brahms known across Europe, declaring this student from Hamburg the coming saviour of German music.

Brahms, meanwhile, was living with Clara and the children - his bedroom on a separate floor, to be sure, but spending most of his time consoling her, helping with the children, and going nearly out of his mind with yearning.

In those years Brahms was slim, beardless and drop-dead handsome. Gossip was sizzling in musical circles. Clara was yearning mightily, too, but as with Brahms her feelings were tangled up with anxiety and guilt. Robert and Clara had been, after all, the supreme musical romance of the Romantic period. Clara was the love of Robert's life, his prime musical champion, the heroic force that had held together his splintering mind longer than anyone could have imagined.

After a protracted decline, Robert died in 1856, whereupon Brahms and Clara were free to declare their passion, to marry. The couple went on holiday to Switzerland to sort it all out. Exactly what he said to her we will never know, but it amounted to this: Cheerio. I'm off to Hamburg. Write if you get work.

Clara put him on the train, staggered home, and told her journal: "I felt as if I were returning from a funeral." Daughter Eugenie later said that Clara could never understand why Brahms so ruthlessly turned away. Clara took up her performing career with a vengeance; it was her solace and, she would tell Brahms, "the very breath of my body".

There are more ironies in this first and greatest, if not precisely last, love of Brahms's life. If he would not marry Clara, neither would he marry anybody else - in his heart he could never leave Clara, nor she him. For the rest of their lives they would maintain their strange but inescapable connection. They spent holidays together. They hugged and kissed.

Their love may never have been consummated. This may seem absurd. But these were different times: no birth control, lots of disease. Proper women shunned affairs. In later years Brahms told an acquaintance that he had never compromised a respectable woman, and for him the definition of a respectable woman was Clara. He once described the aged Clara to a friend thus: "Virginal as ever."

It is hardly a question of Brahms or Clara being sexless. Brahms was famously devoted to prostitutes; for his purposes, he seemed to relegate sex to the professional variety. During their marriage Clara and Robert had maintained a kind of shorthand sexual diary, for medical reasons, which revealed that they were startlingly active throughout. Recall the seven children. And some years later Clara had a brief, unhappy affair with Theodor Kirchner, one of Brahms's best friends. The latter business did not emerge until recent years, and as far as we know Brahms never suspected. At one point he wrote to Clara that Kirchner was talking about killing himself. Never mind, Clara replied, he says that all the time. We may presume this was after the affair.

In short, it was all a splendid mess. What seems to have motivated the rest of Brahms's life, romantic and otherwise, was no more mess. He kept to a life of composing and performing, fought with his friends, tried with imperfect success to keep women at bay, and fled real-life drama whenever it appeared. But the real mess, and a big one, lay inside Brahms himself, in his relations to women and to emotional life in general.

The chaos, the divided nature, likely started at a vulnerable time. At age 13, Brahms was already a phenomenon, with his teachers predicting great things. His parents were supportive, but they were also limited and naive. At some point money was short, so the boy was sent to earn his keep playing piano in some waterfront establishments where his father had worked in his own youth. Maybe these places had been something of a lark to the father. They were not to Brahms.

Popular with sailors, these joints combined the services of bar, restaurant, dance hall and brothel. Brahms was delicately pretty and bad things happened to him in the bars. Mostly he only hinted at what they were (to Clara among others), but for the rest of his life he talked about it, with rage and anguish and sometimes with a fierce pride, for having survived. It steeled him, he said. And this is true. Brahms reached maturity tough as nails.

He said that these experiences ruined women for him. One night, in his cups, he broke up a party by branding all women with a word so sordid that nobody would repeat it. To the friend delegated to sober him up in the park, Brahms raved: "You tell me I should have the same respect, the same exalted homage for women that you have. You expect that of a man cursed with a childhood like mine."

In our terms, German culture at that time was thoroughly misogynistic. Brahms was more so. He relegated sexuality to prostitutes, joked constantly about lady musicians. One woman recalled watching him at a party, stroking his moustache and leering at the girls "as a greedy boy stares at jam tartlets".

Yet if you happened to be female and happened to have talent, there was nobody you would rather have in your corner than Brahms. To a number of women performers he gave extravagantly of his time and money and influence. One was the teenage violinist Marie Soldat, whom he sent off to study with his friend Joachim. When at 20 Soldat made a sensation in Vienna with his Violin Concerto, Brahms was heard shouting from the balcony: "Couldn't she hold her own with 10 men?" Did he love Marie? Probably. How did he squire her? He took her to Vienna's Prater fun fair for the merry-go-round and puppet show.

So Brahms was paradoxically divided about women, as he was about many things. He fell in love periodically after Clara, usually with singers. With the first, Agathe, he exchanged rings, but then sent her a brutal note: "I love you!... But I can't wear chains." Agathe was not the sort you wrote that to. She broke it off and grieved for years. He admitted to friends: "I have played the scoundrel with Agathe."

Brahms understood himself and his divided nature with unsentimental clarity. He was a classicising Romantic, a loner who was a creature of the musical mainstream, a backward-looking artist who anticipated and inspired the future of music. Brahms scorned women and loved them and fled from them, but inescapably needed them.

Whatever the chaos in Brahms's mind ("chaos" his own word, describing his feelings as a teenager), he negotiated his life with extraordinary discipline, common sense, integrity and honesty. The chaos of emotion shackled and subdued by a relentless sense of form and discipline: that is Brahms's art in a nutshell. Likewise, his life. And the most familiar and beloved note in his art is the note of yearning.

· Jan Swafford is a composer and writer on music. His Brahms: a Biography is published by Macmillan.