Isaiah Berlin in 1950: a scholar of incredible breadth, but plagued by self-doubt. Photograph: Getty

Many of us who care about ideas feel a nostalgia for a time when thinkers were taken seriously and when irony was the spice of intellectual life rather than its meat. Few exemplify this mythical age better than the 20th-century historian of ideas Isaiah Berlin. As well as being a scholar of incredible breadth he was also a man of great culture, serving both as president of the British Academy and a director of the Royal Opera House. And yet, as the third volume of his selected letters shows, there need be nothing rarefied about such an authentic life of the mind.

Superficially, Berlin's life was impossibly sophisticated. He dined at the White House and hobnobbed with the likes of Georg Solti, Vladimir Nabokov, Igor Stravinsky and Ezra Pound. And yet no one understood more than Berlin how such a facade can hide the weaknesses, foibles and downright stupidity of human beings. He came to loathe the "hollow virtuosity" that he thought characteristic of his century, as well as the "enormous waste of time at all conferences – the premia on vanity, self-importance, exhibitionism, brilliance in intervention".

He was a flesh and blood intellectual, devoted to his wife, Aline, to whom he wrote: "I love you very hotly & continuously." Even when installed in the high-minded citadel of Harvard he still recognised that their time together provided "the happiest moments of my present life" compared with which "the rest of the civilised life here" was nothing.

He also perceived his own limitations more acutely than his fiercest critics. "It took me years to discover that I was not a philosopher in the proper sense," he wrote. He knew full well that he had never produced a magnum opus: "All I produce is little fragments... Nothing big, nothing solid, nothing genuine."

On that last point he was completely wrong. His essays were authentic and truthful, rejecting simplistic solutions and trying to capture the real complexities of the world. These letters often capture his insights with greater concision and sharpness than the originals.

For instance, it is hard to imagine a clearer precis of his idea of pluralism than the one found in a letter to Dorothea Head, in which he insists "that values collide, that there is no final solution, that one can only gain one value at the expense of another, that whatever one chooses entails the sacrifice of something else – or at any rate often so".

The letters also allow him to clear up a common misconception about the primacy of his idea of negative liberty, "freedom from interference". Berlin thought this extremely important, protesting to Bernard Crick that "you want to identify [freedom] not with the openness of the doors, but with the actual march through them". However, Berlin also insisted that "I do not think that this must be defended at all costs against other values, such as justice, welfare and the development of all kinds of social faculties".

Despite his achievements and abilities, Berlin was plagued with doubts. "I envy the self-confident," he wrote. He was "frightened acutely" as much by success as failure, hating both what he saw as unmerited praise and warranted rejection. "I have not the slightest faith in anything I write myself. It is exactly like money – if you make it yourself it seems a forgery."

Part of the problem seems to be that his balance made it very difficult for him to take clear positions, which made him appear to himself and others as vague or muddled. This is most evident in many of the big political issues of the day, where time and again he was sympathetic to intellectually fashionable causes but would not back them. On Algeria he wrote: "I do not want to subscribe to a positive call for terrorism ... even as a weapon against terrorism." He was unsure about Castro, understood the complexities of the polarised Arab-Israeli dispute, and although he believed getting into Vietnam was a mistake by the US, once there he was worried that pulling out would simply lead to massacres.

In each case, Berlin resisted the allure of cast-iron certainties and moral absolutes. "I remain an unashamed hand-to-mouth pragmatist – one plays the game according to change in circumstances as well as one can and does one's best to catch up with changes in the rules."

As with all honest biographies, these letters expose his vices as well as his virtues. He admits to being "a victim of vanity" on several occasions, such as when accepting the invitation to give the AW Mellon lectures at the National Gallery of Art in Washington in 1965 and the presidency of the British Academy in 1974.

He also seemed to find it hard to live by his avowed belief that "on the whole ... authors should take nasty reviews with a stiff upper lip and just grin and bear it", putting pen to paper a lot in response to criticism.

Inevitably, there is too much in the 700 pages for the casual reader. But even leaving aside the material best reserved for the specialist, there is more here to enlighten and entertain than in the collected works of most of his contemporaries. In their introduction, the editors say: "If this is not one of the best letter-writers of the 20th century, we are ready to eat our respective hats." Gentlemen, you can leave your hats on.