Ed Miliband’s late-night pilgrimage to Russell Brand’s loft apartment, days before the last election, was seen by supporters as a canny bid for the youth vote, and by critics as a cringe-worthy attempt to harness the Shoreditch Messiah’s charisma. Yet neither view captures its real significance as a sign of the profound weakness of mainstream social democracy and its desperate efforts to co-opt the energies of the most dynamic element of today’s left: anarchism. In their eagerness to ridicule Brand’s “ramblings”, commentators have ignored his strong identification with the left-anarchist tradition. For among the works he has recommended to his followers is a collection of writings by another charismatic figure who sometimes lived in London, the father of anarchist communism: Prince Peter Kropotkin.

Comparisons between Kropotkin and Brand may seem strained. Kropotkin’s background as the scion of one of the grandest and most ancient Russian aristocratic families is far removed from Brand’s humbler origins. Kropotkin was a highly educated polymath, while Brand – though undeniably intelligent – has played the part of popular entertainer and motor-mouth wit.

Yet, like Brand, the exiled Kropotkin became a fashionable figure in London, lauded by the late-Victorian artistic and intellectual avant garde – from William Morris to Ford Madox Ford. In a weird prefiguring of the Miliband-Brand wooing, he even received the first Labour Party leader Keir Hardie at his Bromley home. And just as satirical comparisons are made between Brand and the son of God, so Oscar Wilde described Kropotkin as a “beautiful white Christ”.

It is no surprise that anarchist sages and prophets should be so fashionable, both then and now. In Europe before the first world war, those varieties of socialism that placed their faith in state-led social reform – social democracy and Marxism-Leninism – had not yet begun to eclipse their anarchist competitor. But now that era of statist optimism is over, a left reinvigorated by the current crisis of globalised capitalism is searching for alternatives more suited to our individualistic era.

Peter Alexeyevich Kropotkin, born in 1842, came of age in troubled times. Humiliated by his defeat in the Crimean war in 1856, Alexander II set out to reform Russia’s archaic aristocratic order while preserving its fundamentals, and the Kropotkins were stalwarts of the old system. As a youth, Kropotkin trained for the army in Russia’s most elite military academy, and his intellectual distinction even ensured that he was chosen as a page at the tsar’s court. He soon came to despise the status-obsessed cruelties of the ancien regime, identifying not with the nobility but with the peasants who had cared for him as a child.

This alliance of sympathy for the poor with commitment to the life of the mind, especially science, came to define Kropotkin’s career – whether in the service of the tsarist state, or in pursuit of anarchist revolution. Posted to Siberia by the military, he sought to improve the lives of convicts, while also leading pioneering geographical expeditions. And once in exile from Russia (persecuted for his revolutionary activity), he devoted himself to reconciling his deep moral outrage at social inequality with his love of science by developing a coherent anarchist vision – marking him out from his less intellectually ambitious anarchist predecessors, Pierre-Joseph Proudhon and Mikhail Bakunin.

Kropotkin’s synthesis can be found in two of the most important – and readable – texts of anarchism: The Conquest of Bread (1892) and Fields, Factories and Workshops (1899). Society, he argued, could be run along the lines of the peasant communities he saw in Siberia, with their “semi-communistic brotherly organisation”, free of domination by either the state or the market. And this, he insisted, was not mere nostalgia or utopianism, for new technology and modern agriculture would make such decentralised development highly productive. But Kropotkin was mindful, too, of the needs of the environment, an awareness born of his geographical and scientific interests, and he is rightly considered one of the pioneering theorists of Green politics.

He also rooted his anarchism in evolutionary science. Mutual Aid (1902) argued that communities founded on radical equality and participatory democracy were feasible because human nature was innately collaborative. Unlike Social Darwinists such as Herbert Spencer, who argued that all forms of life were driven by a competitive “struggle for existence” between organisms, Kropotkin insisted that another type of struggle was more important – between organisms and the environment. And in this struggle, “mutual aid” was the most effective means of survival.

Between the 1880s and the 1920s, Kropotkin’s communist anarchism competed for influence with a more statist Marxism, and won many converts among intellectuals, peasants and working class communities, especially in southern Europe and the United States (including the “Wobblies” – the Industrial Workers of the World). In Asia, too, anarchism infused the thought of the early Chinese Communist Party, and underpinned the Indian civil disobedience campaigns of Gandhi – though he was closer to Tolstoy’s more religious anarchism.

But the anarchists’ own struggle would be lost, partly because their commitment to democratic participation undercut their capacity to sustain stable mass organisations, and because they were undermined by the violence espoused by some anarchist groups (against Kropotkin’s advice), which provoked ruthless state repression. In the end, though, their fate was sealed by a broader intellectual shift, as the prestige of states rose in the wake of total war – especially in the 1950s and 60s, when both communist east and capitalist west presented rival visions of state-led “progressive modernisation”.

But now states have yet again fallen in popular esteem, damaged by the crisis of Keynesian and communist economics since the 1970s, and by the rise of “60s” values, which prize individual self-expression and personal fulfilment over loyalty to nation states and other centralised institutions.

This individualism is particularly strong among the educated and the young, just as it was among the Bohemians of Victorian England. And it is no surprise that anarchism should have become important again on the left in recent years – from the “anti-globalisers” of the late 1990s, to the 2011 Occupy movement. Indeed Occupy’s principal theorist, David Graeber, is a Kropotkin enthusiast.

Anarchism’s challenges remain much the same as they were in Kropotkin’s day. How can a group so suspicious of established institutions build an effective movement for the long term? How can it win over a majority addicted to endless growth and ever higher living standards? And how can its ideal social order, founded on local participatory democracy, control the enormous concentrations of power in states and international markets?

Yet much has changed to anarchism’s advantage. A more educated society is becoming ever less deferential and possibly less materialistic. Meanwhile, the failures of both state socialism in 1989 and global capitalism in 2008, and their glaring inability to deal with environmental degradation, demand that we question the way we live as never before. Kropotkin is no messiah, but his writings force us to imagine a politics that might just help save the world.

• The Conquest of Bread by Peter Kropotkin is published by Penguin Classics.