"Socialism," George Orwell ranted in The Road to Wigan Pier, manages to draw towards it "every fruit-juice drinker, nudist, sandal-wearer, sex-maniac, Quaker, pacifist and feminist in England". He particularly loathed the "outer-suburban creeping Jesus" readying himself for yoga. But the man Orwell really had it in for was the gay, sandal-making socialist philosopher Edward Carpenter and his Tolstoy-reading, middle-class aficionados, "the sort of eunuch type with a vegetarian smell, who go about spreading sweetness and light."

Orwell was not alone in dishing it out to Comrade Carpenter. The Marxists of the Social Democratic Federation couldn't stand his bourgeois sentimentalities. The Fabians thought him dreadfully naive. And George Bernard Shaw simply dismissed him as "the Noble Savage". But now Sheila Rowbotham, the grande dame of socialist feminist studies, has attempted to lift this left-wing obloquy and save Carpenter from the condescensions of posterity. In the process, she has produced an overlong but magnificent history suffused with scholarship, humanity and a lifetime's feel for the currents of the British progressive tradition. It stands as the definitive Carpenter biography.

As with so many founding fathers of British socialism - Engels and Ruskin among others - Edward Carpenter was born into wealth. He was a child of the mid-Victorian boom, brought up in Brighton on the stock market riches of his father, sharing a happy home with a domineering choir of sisters, mother and aunts. He progressed to Cambridge in the 1870s where two psychological tidal waves hit him: first, his homosexuality and second, "a consciousness of sin" that his family's prosperity had entailed the mass immiseration of the working classes.

Innovatively, Carpenter combined these two sentiments into something approaching a political philosophy. To begin with, he sought refuge in the eroticism of ancient Greek culture (and Rowbotham is masterful at exploring the homosexual currencies of late Victorian Cambridge). But the real breakthrough came with a reading of Walt Whitman and the implicit conjunction of socialism and homosexuality. "Whitman's advocacy of an 'adhesive' democratic manly comradeship was attractive to Carpenter because it provided a new homoerotic possibility and at the same time touched a political nerve," Rowbotham explains. Carpenter then became a vital conduit in bringing Whitman's work before a broader British audience, later visiting him in New Jersey and, so he claimed, sleeping with America's national poet.

After failing to secure his Cambridge fellowship, Carpenter headed north to practise some democratic manly comradeship of his own. A lectureship with the University Extension took him to Sheffield where, in between encounters with "railway-men, porters, clerks, signalmen, ironworkers", he discovered south Yorkshire's sturdier brand of socialism, in contrast to the decadent affluence of Brighton and Cambridge. Now, the exploited had a name and a face and the poverty he witnessed intensified his sense of a civilisation out of sorts.

But Carpenter never subscribed to a socialism of wage levels and work rates. His creed was a "Larger Socialism" which accepted redistribution and land nationalisation, but also drew on the Platonic tradition, German idealism and Hindu spiritualism to offer the sort of grand philosophical solution which so infuriated the Marxists and Fabians. "He wanted a socialism which would not simply end material inequality, but would bring new forms of associating and relating, a new aesthetic of the everyday in harmony with nature," writes Rowbotham. With a libertarian's instinctive distaste for state socialism, Carpenter sought a new way of being human.

An inheritance from his father enabled Carpenter to provide the setting for precisely this. In 1883, he bought three fields at Millthorpe, a tiny, remote settlement nestling in the Cordwell Valley near Sheffield. Run as a co-operative farm, Millthorpe was to witness the Larger Socialism in action as Carpenter and his fellow Simple Lifers started the day with naked swims, worked the earth in woollen tunics and ate a great deal of porridge and radishes. Millthorpe emerged as a countercultural hub in the face of Victorian materialism, becoming an essential stopping-off point for all sorts of confused humanists. Carpenter would sometimes emerge in the morning to find new acolytes wriggling out of overnight mail sacks.

Millthorpe was also renowned for its air of sexual liberation. Carpenter housed a range of boyfriends there, eventually settling on the working-class "rough", George Merrill, but also provided a refuge for troubled souls in a climate of Victorian repression and prudery. Bravely and repeatedly, Carpenter made the case for a liberalisation of homosexual legislation, most notably in Homogenic Love and its Place in a Free Society. After visiting Carpenter (and having his backside touched by Merrill), EM Forster exclaimed in his diary: "Forward rather than back, Edward Carpenter! Edward Carpenter!" and immediately sat down to write his gay novel, Maurice

Carpenter's views on sexuality were partly formed by his engagement with eastern cultures. A keen reader of The Bhagavad Gita, he admired the Hindu religion's worship of sex and travelled extensively around India. He returned to England with not only a commitment to yoga and sandals, but a theosophical conviction in the spiritual side of socialism. His belief in humility in the face of indigenous cultures made him a powerful critic of imperialism in the early 1900s, inspiring anti-colonial activists around the world, including Gandhi.

All of which leads Rowbotham to make the case for Carpenter as a bridge from the narrow-minded Victorians to the emotionally attuned, morally liberal moderns. If so, he was a very antiquated modern. For a large part of Carpenter's legacy was the socialist inclination for a primitivist, Arts and Crafts ruralism that would find its most celebrated outlet in the Garden City movement. Carpenter was the champion of this tradition within the Labour movement and Rowbotham reveals the remarkable extent of his influence over MPs and trade unionists, even as he denounced conscription for the First World War and condemned the growing might of the state.

That innate suspicion of government intervention, born partly of the state's persecution of homosexuals, was why Carpenter was set to be a forgotten prophet. As socialism irrevocably hitched itself to the state, there was ever less space for the truly non-conformist thinking of "cranks" like Carpenter. Bravely, Rowbotham makes a play for Carpenter's place on the left today with the resurgence of environmentalism and quality-of-life concerns, but, in terms of practical achievements, a gay housing co-operative in Sheffield and a playground in north London don't add up to much. In reality, the fire has gone out. Born of spiritualism, socialism is now all state and no spirit. Carpenter the raffish aesthete and rebel ended his days in a bungalow in Guildford.

• Tristram Hunt's Penguin biography of Friedrich Engels will be published in April