On the evening of 30 January 1948, five months after the independence and partition of India, Mohandas Karamchand Gandhi was walking to a prayer meeting at his temporary home in New Delhi when he was shot three times, at point-blank range. He collapsed and died instantly. His assassin, originally feared to be Muslim, turned out to be Nathuram Godse, a Hindu Brahmin from western India. Godse, who made no attempt to escape, said in court that he felt compelled to kill Gandhi since the leader with his womanly politics was emasculating the Hindu nation – in particular, with his generosity to Muslims. Godse is a hero today in an India utterly transformed by Hindu chauvinists – an India in which Mein Kampf is a bestseller, a political movement inspired by European fascists dominates politics and culture, and Narendra Modi, a Hindu supremacist accused of mass murder, is prime minister. For all his talk of Hindu genius, Godse flagrantly plagiarised the fictions of European ethnic-racial chauvinists and imperialists. For the first years of his life he was raised as a girl, with a nose ring, and later tried to gain a hard-edged masculine identity through Hindu supremacism. Yet for many struggling young Indians today Godse represents, along with Adolf Hitler, a triumphantly realised individual and national manhood.

The moral prestige of Gandhi’s murderer is only one sign among many of what seems to be a global crisis of masculinity. Luridly retro ideas of what it means to be a strong man have gone mainstream even in so-called advanced nations. In January Jordan B Peterson, a Canadian self-help writer who laments that “the west has lost faith in masculinity” and denounces the “murderous equity doctrine” espoused by women, was hailed in the New York Times as “the most influential public intellectual in the western world right now”.

This is, hopefully, an exaggeration. It is arguable, however, that a frenetic pursuit of masculinity has characterised public life in the west since 9/11; and it presaged the serial-groping president who boasts of his big penis and nuclear button. “From the ashes of September 11,” the Wall Street Journal columnist Peggy Noonan exulted a few weeks after the attack, “arise the manly virtues.” Noonan, who today admires Peterson’s “tough” talk, hailed the re-emergence of “masculine men, men who push things and pull things”, such as George W Bush, who she half expected to “tear open his shirt and reveal the big ‘S’ on his chest”. Such gush, commonplace at the time, helped Bush, who had initially gone missing in action on 11 September, reinvent himself as a dashing commander-in-chief (and grow cocky enough to dress up as a fighter pilot and compliment Tony Blair’s “cojones”).

As manly virtues arose, attacks on women, and feminists in particular, in the west became nearly as fierce as the wars waged abroad to rescue Muslim damsels in distress. In Manliness (2006) Harvey Mansfield, a political philosopher at Harvard, denounced working women for undermining the protective role of men. The historian Niall Ferguson, a self-declared neo-imperialist, bemoaned that “girls no longer play with dolls” and that feminists have forced Europe into demographic decline. More revealingly, the few women publicly critical of the bellicosity, such as Katha Pollitt, Susan Sontag and Arundhati Roy, were “mounted on poles for public whipping” and flogged, Barbara Kingsolver wrote, with “words like bitch and airhead and moron and silly”. At the same time, Vanity Fair’s photo essay on the Bush administration at war commended the president for his masculine sangfroid and hailed his deputy, Dick Cheney, as “The Rock”.

Psychotic masculinity can be seen everywhere from ISIS to mass-murderer Anders Breivik, who claimed Viking ancestry

Some of this post-9/11 cocksmanship was no doubt provoked by Osama bin Laden’s slurs about American manhood: that the free and the brave had gone “soft” and “weak”. Humiliation in Vietnam similarly brought forth such cartoon visions of masculinity as Sylvester Stallone and Arnold Schwarzenegger. It is also true that historically privileged men tend to be profoundly disturbed by perceived competition from women, gay people and diverse ethnic and religious groups. In Sexual Anarchy: Gender and Culture at the Fin de Siecle (1990) Elaine Showalter described the great terror induced among many men by the very modest gains of feminists in the late 19th century: “fears of regression and degeneration, the longing for strict border controls around the definition of gender, as well as race, class and nationality”.

In the 1950s, historian Arthur Schlesinger Jr was already warning of the “expanding, aggressive force” of women, “seizing new domains like a conquering army”. Exasperated by the “castrated” American male and his “feminine fascination for the downtrodden”, Schlesinger, the original exponent of muscular liberalism, longed for the “frontiersmen” of American history who “were men, and it did not occur to them to think twice about it”.

These majestically male makers of the modern west are being forced to think twice about a lot today. Gay men and women are freer than before to love whom they love, and to marry them. Women expect greater self-fulfilment in the workplace, at home and in bed. Trump may have the biggest nuclear button, but China leads in artificial intelligence as well as old-style mass manufacturing. And technology and automation threaten to render obsolete the men who push and pull things – most damagingly in the west.

Many straight white men feel besieged by “uppity” Chinese and Indian people, by Muslims and feminists, not to mention gay bodybuilders, butch women and trans people. Not surprisingly they are susceptible to Peterson’s notion that the ostensible destruction of “the traditional household division of labour” has led to “chaos”. This fear and insecurity of a male minority has spiralled into a politics of hysteria in the two dominant imperial powers of the modern era. In Britain, the aloof and stiff upper-lipped English gentleman, that epitome of controlled imperial power, has given way to such verbally incontinent Brexiters as Boris Johnson. The rightwing journalist Douglas Murray, among many elegists of English manhood, deplores “emasculated Italians, Europeans and westerners in general” and esteems Trump for “reminding the west of what is great about ourselves”. And, indeed, whether threatening North Korea with nuclear incineration, belittling people with disabilities or groping women, the American president confirms that some winners of modern history will do anything to shore up their sense of entitlement.

But gaudy displays of brute manliness in the west, and frenzied loathing of what the alt-rightists call “cucks” and “cultural Marxists”, are not merely a reaction to insolent former weaklings. Such manic assertions of hyper-masculinity have recurred in modern history. They have also profoundly shaped politics and culture in Asia, Africa and Latin America. Osama bin Laden believed that Muslims “have been deprived of their manhood” and could recover it by obliterating the phallic symbols of American power. Beheading and raping innocent captives in the name of the caliphate, the black-hooded young volunteers of Islamic State were as obviously a case of psychotic masculinity as the Norwegian mass-murderer Anders Behring Breivik, who claimed Viking warriors as his ancestors. Last month, the Philippines president Rodrigo Duterte told female rebels in his country that “We will not kill you. We will just shoot you in the vagina.” Tormenting hapless minorities, India’s Hindu supremacist chieftains seem obsessed with proving, as one asserted after India’s nuclear tests in 1998, “we are not eunuchs any more”.

Morbid visions of castration and emasculation, civilisational decline and decay, connect Godse and Schlesinger to Bin Laden and Trump, and many other exponents of a rear-guard machismo today. They are susceptible to cliched metaphors of “soft” and “passive” femininity, “hard” and “active” masculinity; they are nostalgic for a time when men did not have to think twice about being men. And whether Hindu chauvinist, radical Islamist or white nationalist, their self-image depends on despising and excluding women. It is as though the fantasy of male strength measures itself most gratifyingly against the fantasy of female weakness. Equating women with impotence and seized by panic about becoming cucks, these rancorously angry men are symptoms of an endemic and seemingly unresolvable crisis of masculinity.

When did this crisis begin? And why does it seem so inescapably global? Writing Age of Anger: A History of the Present, I began to think that a perpetual crisis stalks the modern world. It began in the 19th century, with the most radical shift in human history: the replacement of agrarian and rural societies by a volatile socio-economic order, which, defined by industrial capitalism, came to be rigidly organised through new sexual and racial divisions of labour. And the crisis seems universal today because a web of restrictive gender norms, spun in modernising western Europe and America, has come to cover the remotest corners of the earth as they undergo their own socio-economic revolutions.

There were always many ways of being a man or a woman. Anthropologists and historians of the world’s astonishingly diverse pre-industrial societies have consistently revealed that there is no clear link between biological makeup and behaviour, no connection between masculinity and vigorous men, or femininity and passive women. Indians, British colonialists were disgusted to find, revered belligerent and sexually voracious goddesses, such as Kali; their heroes were flute-playing idlers such as Krishna. A vast Indian literature attests to mutably gendered men and women, elite as well as folk traditions of androgyny and same-sex eroticism.

These unselfconscious traditions began to come under unprecedented assault in the 19th century, when societies constituted by exploitation and exclusion, and stratified along gender and racial lines, emerged as the world’s most powerful; and when such profound shocks of modernity as nation-building, rural-urban migration, imperial expansion and industrialisation drastically changed all modes of human perception. A hierarchy of manly and unmanly human beings had long existed in many societies without being central in them. During the 19th century, it came to be universally imposed, with men and women straitjacketed into specific roles.

‘In the 19th century, the ideal of a strong, fearless manhood came to be embodied in muscular selves, nations, empires and races’

The modern west appears, in the western supremacist version of history, as the guarantor of equality and liberty to all. In actuality, a notion of gender (and racial) inequality, grounded in biological difference, was, as Joan Wallach Scott demonstrates in her recent book Sex and Secularism, nothing less than “the social foundation of modern western nation-states”. Immanuel Kant dismissed women as incapable of practical reason, individual autonomy, objectivity, courage and strength. Napoleon, the child of the French Revolution and the Enlightenment, believed women ought to stay at home and procreate; his Napoleonic Code, which inspired state laws across the world, notoriously subordinated women to their fathers and husbands. Thomas Jefferson, America’s founding father, commended women, “who have the good sense to value domestic happiness above all other” and who are “too wise to wrinkle their foreheads with politics”. Such prejudices helped replace traditional patriarchy with the exclusionary ideals of masculinity as the modern world came into being.

On such grounds, women were denied political participation and forced into subordinate roles in the family and the labour market. Pop psychologists periodically insist that men are from Mars and women from Venus, lamenting the loss of what Peterson calls “traditional” divisions of labour, without acknowledging that capitalist, industrial and expansionist societies required a fresh division of labour, or that the straight white men who supervised them deemed women unfit, due to their physical or intellectual inferiority, to undertake territorial aggrandisement, nation-building, industrial production, international trade, and scientific innovation. Women’s bodies were meant to reproduce and safeguard the future of the family, race and nation; men’s were supposed to labour and fight. To be a “mature” man was to adjust oneself to society and fulfil one’s responsibility as breadwinner, father and soldier. “When men fear work or fear righteous war,” as Theodore Roosevelt put it, “when women fear motherhood, they tremble on the brink of doom.” As the 19th century progressed, many such cultural assumptions about male and female identity morphed into timeless truths. They are, as Peterson’s rowdy fan club reveals, more vigorously upheld today than the “truths” of racial inequality, which were also simultaneously grounded in “nature”, or pseudo-biology.

Scott points out that the modes of sexual difference defined in the modernising west actually helped secure, “the racial superiority of western nations to their ‘others’ – in Africa, Asia, and Latin America”. “White skin was associated with ‘normal’ gender systems, dark skin with immaturity and perversity.” Thus, the British judged their Kali-worshipping Indian subjects to be an unmanly and childish people who ought not to wrinkle their foreheads with ideas of self-rule. The Chinese were widely seen, including in western Chinatowns, as pigtailed cowards. Even Muslims, Christendom’s formidable old rivals, came to be derided as pitiably “feminine” during the high noon of imperialism.

Gandhi explicitly subverted these gendered prejudices of European imperialists (and their Hindu imitators): that femininity was the absence of masculinity. Rejecting the western identification of rulers with male supremacy and subjecthood with feminine submissiveness, he offered an activist politics based on rigorous self-examination and maternal tenderness. This rejection eventually cost him his life. But he could see how much the male will to power was fed by a fantasy of the female other as a regressive being – someone to be subdued and dominated – and how much this pathology had infected modern politics and culture.

Its most insidious expression was the conquest and exploitation of people deemed feminine, and, therefore, less than human – a violence that became normalised in the 19th century. For many Europeans and Americans, to be a true man was to be an ardent imperialist and nationalist. Even so clear-sighted a figure as Alexis de Tocqueville longed for his French male compatriots to realise their “warlike” and “virile” nature in crushing Arabs in north Africa, leaving women to deal with the petty concerns of domestic life.

As the century progressed, the quest for virility distilled a widespread response among men psychically battered by such uncontrollable and emasculating phenomena as industrialisation, urbanisation and mechanisation. The ideal of a strong, fearless manhood came to be embodied in muscular selves, nations, empires and races. Living up to this daunting ideal required eradicating all traces of feminine timidity and childishness. Failure incited self-loathing – and a craving for regenerative violence. Mocked with such unmanly epithets as “weakling” and “Oscar Wilde”, Roosevelt tried to overcome, Gore Vidal once pointed out, “his physical fragility through ‘manly’ activities of which the most exciting and ennobling was war”. It is no coincidence that the loathing of homosexuals, and the hunt for sacrificial victims such as Wilde, was never more vicious and organized than during this most intense phase of European imperialism.

One image came to be central to all attempts to recuperate the lost manhood of self and nation: the invincible body, represented in our own age of extremes by steroid-juiced, knobbly musculature. Actually, size matters today much less than it ever did; not many muscles are required for increasingly sedentary work habits and lifestyles. Nevertheless, an obsession with raw brawn and sheer mass still shapes political cultures. Trump’s boasts about the size of his body parts were preceded by Vladimir Putin’s displays of his pectorals – advertisements for a Russia re-masculinised after its emasculation by Boris Yeltsin, a flabby drunk. But shirtless hunks are also a striking recent phenomenon in Godse’s “rising” India. In the 90s, just as India’s Hindu nationalisation got into gear, formerly scrawny or chubby Bollywood stars began to flaunt glisteningly hard abs and bulging biceps; Rama, the lean-limbed hero of the Ramayana, started to resemble Rambo in calendar art and political posters. These buffed-up bodies of popular culture foreshadowed Modi, who rose to power boasting of his 56-inch chest, and promising true national potency to young unemployed stragglers.

This vengeful masculinist nationalism was the original creation of Germans in the early 19th century, who first outlined a vision of creating a superbly fit people or master race and fervently embraced such typically modern forms of physical exercise as gymnastics, callisthenics and yoga and fads like nudism. But pumped-up anatomy emerged as a “natural” embodiment of the evidently exclusive male virtue of strength only as the century ended. As societies across the west became more industrial, urban and bureaucratic, property-owning farmers and self-employed artisans rapidly turned into faceless office workers and professionals. With “rational calculation” installed as the new deity, “each man”, Max Weber warned in 1909, “becomes a little cog in the machine”, pathetically obsessed with becoming “a bigger cog”. Increasingly deprived of their old skills and autonomy in the iron cage of modernity, working class men tried to secure their dignity by embodying it in bulky brawn.

Historians have emphasised how male workers, humiliated by such repressive industrial practices as automation and time management, also began to assert their manhood by swearing, drinking and sexually harassing the few women in the workforce – the beginning of an aggressive hardhat culture that has reached deep into blue-collar workplaces during the decades-long reign of neoliberalism. Towards the end of the 19th century large numbers of men embraced sports and physical fitness, and launched fan clubs of pugnacious footballers and boxers.

It wasn’t just working men. Upper-class parents in America and Britain had begun to send their sons to boarding schools in the hope that their bodies and moral characters would be suitably toughened up in the absence of corrupting feminine influences. Competitive sports, which were first organised in the second half of the 19th century, became a much-favoured means of pre-empting sissiness – and of mass-producing virile imperialists. It was widely believed that putative empire-builders would be too exhausted by their exertions on the playing fields of Eton and Harrow to masturbate.

But masculinity, a dream of power, tends to get more elusive the more intensely it is pursued; and the dread of emasculation by opaque economic, political and social forces continued to deepen. It drove many fin de siècle writers as well as politicians in Europe and the US into hyper-masculine trances of racial nationalism – and, eventually, the calamity of the first world war. Nations and races as well as individuals were conceptualised as biological entities, which could be honed into unassailable organisms. Fear of “race suicide”, cults of physical education and daydreams of a “New Man” went global, along with strictures against masturbation, as the inflexible modern ideology of gender difference reached non-western societies.

European colonialists went on to impose laws that enshrined their virulent homophobia and promoted heterosexual conjugality and patrilineal orders. Their prejudices were also entrenched outside the west by the victims of what the Indian critic Ashis Nandy calls “internal colonialism”: those subjects of European empires who pleaded guilty to the accusation that they were effeminate, and who decided to man up in order to catch up with their white overlords.

This accounts for a startling and still little explored phenomenon: how men within all major religious communities – Buddhist, Hindu and Jewish as well as Christian and Islamic – started in the late 19th century to simultaneously bemoan their lost virility and urge the creation of hard, inviolable bodies, whether of individual men, the nation or the umma. These included early Zionists (Max Nordau, who dreamed of Muskeljudentum, “Jewry of Muscle”), Asian anti-imperialists (Swami Vivekananda, Modi’s hero, who exhorted Hindus to build “biceps”, and Anagarika Dharmapala, who helped develop the muscular Buddhism being horribly flexed by Myanmar’s ethnic-cleansers these days) as well as fanatical imperialists such as Robert Baden-Powell, founder of the Scout movement.

The most lethal consequences of this mimic machismo unfolded in the first decades of the 20th century. “Never before and never afterwards”, as historian George Mosse, the pioneering historian of masculinity, wrote, “has masculinity been elevated to such heights as during fascism”. Mussolini, like Roosevelt, transformed himself from a sissy into a fire-breathing imperialist. “The weak must be hammered away,” declared Hitler, another physically ill-favoured fascist. Such wannabe members of the Aryan master race accordingly defined themselves against the cowardly Jew and discovered themselves as men of steel in acts of mass murder.

This hunt for manliness continues to contaminate politics and culture across the world in the 21st century. Rapid economic, social and technological change in our own time has plunged an exponentially larger number of uprooted and bewildered men into a doomed quest for masculine certainties. The scope for old-style imperialist aggrandisement and forging a master race may have diminished. But there are, in the age of neoliberal individualism, infinitely more unrealised claims to masculine identity in grotesquely unequal societies around the world. Myths of the self-made man have forced men everywhere into a relentless and often futile hunt for individual power and wealth, in which they imagine women and members of minorities as competitors. Many more men try to degrade and exclude women in their attempt to show some mastery that is supposed to inhere in their biological nature.

Fear of femini​​sation​​ has driven demagogic movements like that unleashed by the locker room bully in the White House

Frustration and fear of feminisation have helped boost demagogic movements similar to the one unleashed by the locker room bully in the White House. Godse’s hyper-masculine cliches have vanquished the traditions of androgyny that Gandhi upheld – and not just in India. Young Pakistani men revere the playboy-turned-politician Imran Khan as their alpha male redeemer; they turn viciously on critics of his indiscretions. Similarly embodying a triumphant masculinity in the eyes of his followers, the Turkish president Recep Tayyip Erdoğan can do no wrong. Rodrigo Duterte jokes, with brazen frequency, about rape.

Misogyny now flourishes in the public sphere because, as in modernising Europe and America, many toilers daydream of a primordial past when real men were on top, and women knew their place. Loathing of “liberated” women who seem to be usurping male domains is evident not only on social media but also in brutal physical assaults. These are sanctioned by pseudo-traditional ideologies such as Hindu supremacism and Islamic fundamentalism that offer to many thwarted men in Asia and Africa a redeeming machismo: the gratifying replacement of neoliberalism’s bogus promise of equal opportunity with old-style patriarchy.

Susan Faludi argues that many Americans used the 9/11 attacks to shrink the gains of feminism and push women back into passive roles. Peterson’s traditionalism is the latest of many attempts in the west in recent years to restore the authority of men, or to remasculinise society. These include the deployment of “shock-and-awe” violence, loathing of cucks, cultural Marxists and feminists, re-imagining a silver-spooned posturer like Bush as superman, and, finally, the political apotheosis of a serial groper.

This recurrent search for security in coarse manhood confirms that the history of modern masculinity is the history of a fantasy. It describes the doomed quest for a stable and ordered world that entails nothing less than war on the irrepressible plurality of human existence – a war that is periodically renewed despite its devastating failures. An outlandish phobia of women and effeminacy may be hardwired into the long social, political and cultural dominance of men. It could be that their wounded sense of entitlement, or resentment over being denied their customary claim to power and privilege, will continue to make many men vulnerable to such vendors of faux masculinity as Trump and Modi. A compassionate analysis of their rage and despair, however, would conclude that men are as much imprisoned by man-made gender norms as women.

“One is not born, but rather becomes a woman” wrote Simone de Beauvoir. She might as well have said the same for men. “It is civilisation as a whole that produces such a creature.” And forces him into a ruinous pursuit of power. Compared with women, men are almost everywhere more exposed to alcoholism, drug addiction, serious accidents and cardiovascular disease; they have significantly lower life expectancies and are more likely to kill themselves. The first victims of the quest for a mythical male potency are arguably men themselves, whether in school playgrounds, offices, prisons or battlefields. This everyday experience of fear and trauma binds them to women in more ways than most men, trapped by myths of resolute manhood, tend to acknowledge.

Certainly, men would waste this latest crisis of masculinity if they deny or underplay the experience of vulnerability they share with women on a planet that is itself endangered. Masculine power will always remain maddeningly elusive, prone to periodic crises, breakdowns and panicky reassertions. It is an unfulfillable ideal, a hallucination of command and control, and an illusion of mastery, in a world where all that is solid melts into thin air, and where even the ostensibly powerful are haunted by the spectre of loss and displacement. As a straitjacket of onerous roles and impossible expectations, masculinity has become a source of great suffering – for men as much as women. To understand this is not only to grasp its global crisis today. It is also to sight one possibility of resolving the crisis: a release from the absurd but crippling fear that one has not been man enough •

•Pankaj Mishra’s Age of Anger: A History of the Present is published in paperback by Penguin. To order a copy for £8.49 (RRP £9.99) go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.