‘A means to counter the sorry threadbareness of imperial Britain’: mods in Brighton, 1966. Photograph: David Mcenery/Rex Features

As West Indian migrants sought a new and better life in 1960s Britain, mods began to listen to Jamaican ska, a speedy jazz-tinged shuffle beat. Ska had originated in the early 1960s as a Jamaican twist on black American dance music ("upside-down R&B", the Jamaican reggae guitarist Ernest Ranglin called it). Scooter-riding mods adopted the music as a supplement to their diet of imported American soul and jazz, and soon a taste of Trench Town swagger was brought to inner-city Britain, through such Jamaican groups as the Skatalites, and the Jamaican singers Derrick Morgan and the late, great Desmond Dekker.

Ska was, triumphantly, a Commonwealth music that took hold in Britain's inner cities where Caribbean migration was most dense. My Boy Lollipop, sung by Millie Small with a pert underage suggestiveness, was one of the earliest pop-ska hits. It swept Britain in 1964, and became the year's bestselling single ahead of the Beatles and the Stones. For a while this sort of music brought urban whites and blacks together. By the early 1970s, however, with football "hooliganism" a fact of British life, mod culture had splintered into its skinhead offshoot, which was often seen as National Front fodder. Enoch Powell had evoked images of a race war in his 1968 "rivers of blood" speech, and Britain no longer seemed so tolerant of minorities.

Oddly, given their perceived racial antagonisms, skinheads rarely beat up West Indians (by "niggers" they usually meant Asians, who were seen to be easy targets and lamentably unhip). Ska's driving, dancefloor rhythms appealed to the suedehead's passion for uptempo black music, sharp clothes and short hair. Sometimes, if suitably dressed in Crombies and Trevira suits, Jamaicans were allowed to join skinhead gangs and drink cans of Harp lager with them on Saturday nights in the dancehall (Red Stripe came later). West Indians were seen as less satisfactory when they "acted clannish" or "kept to themselves": in other words, when they failed – like the despised "Pakis" – to assimilate into British culture.

Mod's first choice of music was jazz, Richard Weight reminds us. Miles Davis in particular became a fashion icon for blue-eyed soul brothers everywhere in Britain. The photograph of Davis on the cover of his celebrated 1958 Milestones album – Sta-Prest trousers, button-down Ivy League shirt – became a sort of mod pin-up. Mods ("modernists") were among the first white Britons to embrace west-coast jazz, which had been galvanized by the Birth of the Cool sessions led by Davis in New York from 1949-50.

Soho jazz clubs such as the Flamingo, where Jamaican ska-reggae bands played alongside mohair-clad "modernists" such as Georgie Fame, helped to break down race prejudice in postwar Britain and provided white audiences with an opportunity to encounter West Indians and even (heavens!) talk to them. In Weight's analysis, mod was a uniquely British amalgam of black American and black Jamaican music and European fashions. Soho, where Italians had settled after the war in order to work in ice-cream and confectionery, was a haven of continental mod styles ranging from Gaggia espresso machines to Vespa scooters. The Italian film star Marcello Mastroianni, impeccably dressed in Fellini's La Dolce Vita, was a mod icon, whose world-weary allure and curiously impotent sex appeal conformed to a mod ideal of dandyish (perhaps even homosexual) urbanity.

If mods disdained rockers, says Weight, it was because they mimicked aspects of American popular culture without "creating anything original". Mod, the first distinctively British youth culture, thrived at a time when the empire was played out and moribund; the appropriation of RAF insignia and the union flag was a means to counter the sorry threadbareness of imperial Britain. After the material deprivations of the postwar years (when, as we read in Ian Fleming's Dr No, "people streamed miserably to work, their legs whipped by the wet hems of their macintoshes") came the mod extravagance of Carnaby Street, Mary Quant and Terence Conran's first Habitat store.

Conran was, in a sense, an early mod, who brought a taste of continental design (not to mention duvets) to a newly affluent British public. Weight dilates knowledgeably on mod-inspired films, architecture and design, as well as music. As the 1950s gave way to the 1960s, the taste for Miles Davis and Thelonius Monk was replaced by parkas, amphetamines and Who albums. (Pete Townshend appeared on the cover of the Observer magazine in 1966 wearing a jacket stitched out of the union colours.) Another very English mod band, the Kinks, offered a hybrid of music hall and electrified Noël Coward. Waterloo Sunset, a classic Ray Davies weepy, shows a mod fascination in the vagaries of the English class system and a roseate glow of romance.

Mod was revived in the late 1970s in the Midlands by 2 Tone bands such as the Specials and the Selecter, who fused a pavement-pounding leftist politics with the jolly-up jazz of Jamaican ska. In Coventry and Birmingham it was again cool for whites to have black friends; ska-loving "Cov lads" and their rude boy allies helped to make black West Indian culture synonymous with youth culture in Britain today, where a Jamaican inflection has long been hip even among white teenagers. Mod, well-written throughout, crackles with reflections on fashion, music and film, as these became the giant pop art project of the far-distant 1960s and beyond.

Ian Thomson's The Dead Yard: A Story of Modern Jamaica is published by Faber