In May 1992, the dons of Cambridge University filed into their parliament to vote on whether to award an honorary degree to the French philosopher Jacques Derrida, founder of so-called deconstruction. Despite a deftly managed smear campaign by the opposition, Derrida's supporters carried the day. It would be interesting to know how many of those who tried to block him in the name of rigorous scholarship had read a single book of his, or even a couple of articles.

The truth is that they did not need to. The word was abroad that this purveyor of fashionable French gobbledegook was a charlatan and a nihilist, a man who believed that anything could mean anything and that there was nothing in the world but writing. He was a corrupter of youth who had to be stopped in his tracks. As a teenager, Derrida had fantasised with some of his friends about blowing up their school with some explosives they had acquired. There were those in Cambridge who thought he was planning to do the same to western civilisation. He did, however, have an unlikely sympathiser. When the Duke of Edinburgh, chancellor of Cambridge University, presented Derrida with his degree in the year in which Charles and Diana separated, he murmured to him that deconstruction had begun to affect his own family too.

Deconstruction holds that nothing is ever entirely itself. There is a certain otherness lurking within every assured identity. It seizes on the out-of-place element in a system, and uses it to show how the system is never quite as stable as it imagines. There is something within any structure that is part of it but also escapes its logic. It comes as no surprise that the author of these ideas was a Sephardic Jew from colonial Algeria, half in and half out of French society. If his language was French, he could also speak the patois of working-class Arabs. He would later return to his home country as a conscript in the French army, a classic instance of divided identity.

At the age of 12, Derrida was excluded from his lycee when the Algerian government, anxious to outdo the Vichy regime in its antisemitic zeal, decided to lower the quota of Jewish pupils. Paradoxically, the effect of this brutal rejection on a "little black and very Arab Jew", as he described himself, was not only to make him feel an outsider, but to breed in him a lifelong aversion to communities. He was taken in by a Jewish school, and hated the idea of being defined by his Jewish identity. Identity and homogeneity were what he would later seek to deconstruct. Yet the experience also gave him a deep suspicion of solidarity.

If he was always a man of the left, he had an outsider's distaste for orthodoxy and organisation. His role was that of the gadfly, the professional dissident, the joker in the pack. In the end, he was writing of the "absolute singularity" of every human being, and was always a dedicated non-joiner. Norms, doctrines and mass movements were likely to be oppressive, whereas margins and deviations were potentially subversive. Yet the English Defence League is marginal. And it took a mass movement to topple Gaddafi. Respecting freedom of speech is an orthodoxy, and the right to strike is a doctrine.

From a modest background in Algiers, Derrida moved to the most prestigious lycee in France, and from there to the Ecole Normale Supérieure. It was a heavily Stalinist institution at the time, which confirmed his reluctance to shout with the larger crowd. If Derrida was later to declare himself a communist, it was only in the sense that Kennedy called himself a Berliner. When student revolt erupted around him in May 1968, he stood mostly on the sidelines. Yet the libertarian impulse of the sixty-eighters was also a driving force behind his own work. The previous year had been his annus mirabilis, witnessing the appearance of three of the books that were to make his name revered and reviled across the globe.

Before long, the taciturn, socially gauche young man from the colonies was gracing the dinner tables of a galaxy of French luminaries: Jean Genet, Roland Barthes, Julia Kristeva, Maurice Blanchot and others. Even the French government fell under his spell. When François Mitterrand came to power in 1981, Derrida was invited to set up an international college of philosophy in Paris. Deconstruction was now all the rage from Sydney to San Diego, while Derrida himself was feted as an intellectual superstar. Soon, there was an American comic book featuring a sinister Doctor Deconstructo, and magazines on home decor were inviting their readers to deconstruct the concept of a garden.

I suspect that one reason Derrida enjoyed travelling the world so much was because it allowed him some respite from the bitchy, sectarian, backstabbing, backscratching climate of Parisian intellectual life, which this superb biography faithfully records. What the book fails to underline quite as heavily is how waspish the maitre himself could be.

Two dramatic moments stand out in Derrida's subsequent career. Travelling to communist Prague in 1981 to address a secretly organised philosophy seminar, he was arrested and charged with drug smuggling. It seems the authorities saw the dismantling of binary oppositions as a threat to the state. The police officer who had planted the drugs in Derrida's suitcase was himself later arrested for drug trafficking.

Six years later, Derrida's life was again disrupted, this time by the revelation that his recently dead friend, the critic Paul de Man, had contributed antisemitic articles to the pro-German Belgian press during the second world war. Shattered by the news, Derrida wrote a long essay in De Man's defence – which must rank among the most shamelessly disingenuous texts of modern times.

Benoît Peeters has ransacked the voluminous Derrida archives and interviewed scores of his friends and colleagues. The result is a marvellously compelling account, lucidly translated by Andrew Brown. The man who emerges from this portrait is an agonised soul with sudden outbreaks of gaiety, an astonishingly original thinker with more than a dash of vanity who nevertheless made himself fully available to the humblest student.

In personal conversation he was that most admirable of intellectuals, one visibly relieved not to have to speak of intellectual matters. He was one of the latest in an honourable lineage of anti-philosophers – from Kierkegaard, Nietzsche and Marx to Freud, Adorno, Wittgenstein and Walter Benjamin – who could say what they had to say only by inventing a new style of writing and philosophising.

Not all of Derrida's writing is to everyone's taste. He had an irritating habit of overusing the rhetorical question, which lends itself easily to parody: "What is it, to speak? How can I even speak of this? Who is this 'I' who speaks of speaking?"

Even so, the Cambridge backwoodsmen were wrong. Derrida, who died of cancer in 2004 urging his friends to affirm life, was no nihilist. Nor did he want to blow up western civilisation with a stick of conceptual dynamite. He simply wished to make us less arrogantly assured that when we speak of truth, love, identity and authority, we know exactly what we mean.

• Terry Eagleton's Why Marx Was Right is published by Yale University Press.