On 8 November, the day before the opening of the blockbuster Leonardo da Vinci exhibition at the National Gallery, Sky Arts broadcast an 80-minute film about the show. Charles Nicholl, whose acclaimed life of Leonardo was published in 2004, "was supposedly in this rather honourable position of being the last person to talk on the programme". But because the programme over-ran, he "ended up sounding like I was the only person in the entire National Gallery" who didn't think the recently attributed Christ As Salvator Mundi was by Leonardo, "which wasn't quite what I wanted to say. But I did manage to blurt out before the credits rolled over the top of me that one shouldn't be railroaded into these things."

Sitting with Nicholl in his Tuscan farmhouse, up in the hills not far outside the walled medieval city of Lucca – on a clear day, he says, you can just about see Vinci from his front garden – I find it hard to imagine him being railroaded into anything. His conversation, like his writing, follows where his curiosity leads, and it's as likely to take him off up an unmarked goat track as along more familiar routes. He's variously described as a historian, biographer and travel writer, though most of his work doesn't fall straightforwardly into any of those categories. His 12th and most recent book, Traces Remain, published this month, is a collection of essays written over the past two decades. The subjects range from Thomas Coryate and Edward Kelley to Arthur Cravan and Jim Thompson. (And no, if it weren't for Nicholl I'd never have heard of them either.) They're slightly shady, marginal characters, who existed in two or more worlds at once but ultimately belonged to none. The same could be said of Nicholls's books, which occupy an uncertain but appealing ground between academia and journalism.

Coryate, c1577-1617, was "a kind of comedian, a learned buffoon, a butt for courtly wits and poets such as John Donne and Ben Jonson", as well as "an immensely tough and courageous traveller", who walked across much of Europe and Asia. Kelley was an English "alchemist, clairvoyant and conman" who died in a Bohemian prison in the 1590s. Cravan was a French boxer and poet who disappeared in Mexico in 1918, Thompson a British silk tycoon who disappeared in the Malaysian jungle in 1967. Nicholl has followed their trails both literally and figuratively, on the ground and in the archives, displaying a remarkable eye and ear for the telling detail, until those trails run cold: "Almost everything about Coryate's last journey is uncertain," he writes. "Its precise route cannot be traced, its circumstances cannot be recovered … One glimpses him out of dusty bus windows: a ragged man walking alone down a road."

Biography as Nicholl writes it is also a form of travel writing. "The same things attract me to the Elizabethan back streets as attracted me to the back streets of Bogotá," he says, "though in the Elizabethan ones you travel a little more safely." To look for Coryate he travelled to India, to Mexico for Cravan, to the Czech Republic after Kelley. "I feel the travel book is a bit of a played-out genre," he says. Biography offers a new kind of map: "When you go off on one of these biographical research trips, those are the untrodden paths, that's what I love about it: you end up in places that as a traveller you would never intend to go."

Even when pursuing more famous quarry – as well as Leonardo, he's written books about Shakespeare, Marlowe and Rimbaud – Nicholl prefers to approach them from an oblique angle. The Lodger (2007) comes to Shakespeare by way of a court case involving his old landlord, Christopher Mountjoy of Silver Street in Cripplegate. The Mountjoys' maid refers in her deposition to "one Mr Shakespeare that laye in the house". According to Nicholl: "I think it was the marvellous banality of this phrase that first sparked my interest in the case. For a moment we see him not from the viewpoint of literary greatness." In the introduction to his biography of Leonardo, he writes that "it is the task of this book to try to recover something of Leonardo the man – that is, Leonardo the real man, who lived in real time and ate real bowls of soup, as opposed to Leonardo the superhuman, multi-disciplinary 'Universal Man' with whom we are more habitually presented."

Nicholl is, in his unassuming way, a dedicated iconoclast – committed not to denying the greatness of Shakespeare or Leonardo's work, but to demonstrating that great work is produced by ordinary people. His enduring subject, whoever he's writing about, is "the realities of life as it was lived by that person". "There's one world that you know these people very well from," he says, "so let's have a look at the other one, at the other, dark side of the moon, as it were: Marlowe as spy, Rimbaud as traveller or explorer and gun-runner, Shakespeare as lodger rather than great playwright." Hence his impatience with the "air of reverence" he felt at the National Gallery. "The presenter of Sky Arts had used the phrase 'I'm a believer' just before my segment came on, so I was reacting against that," he says. "I suspect I was being a pure contrarian … Leonardo himself always believed in scepti cism as the first reaction."

One of the essays in Traces Remain is on Leonardo's notebooks. Nicholl quotes from "a list of things-to-do in a Milanese notebook of c1508": "Describe how the clouds are formed and how they dissolve … Describe what sneezing is, what yawning is … Describe the tongue of the woodpecker." Nicholl's own "always growing list" is almost as diverse. It seems to grow even as he's talking to me, mentioning a new idea every 10 minutes.

His next book will be about Francis Barber, Dr Johnson's Jamaican valet. Then, "one of my next big projects, if I ever do another big project, is a book about Shakespeare entirely studied through the contemporary and near-contemporary accounts of him."

And then there's the matter of Ben Jonson's chair, of which he thinks he's found an image: "John Aubrey describes having seen Ben Jonson's chair. Whether Aubrey's telling the truth or whether the person showing it to him is telling the truth … I'd better not say too much about it because someone else will nick the idea." Well, possibly, though it's hard to imagine anyone else making a decent story out of it. Part of the appeal of Leonardo's notebooks, Nicholl has written, is that they are "full of physical presence, and of things – the aniseed gobstoppers, the Turkish leather, the purse on the bed, the little flasks of oil broken on the floor". That sense of "physical presence" is what appeals to him about Jonson's chair: it's where he sat to write his plays.

Aubrey, who died in 1697 and is best known for his Brief Lives, clearly a model for Nicholl's own "biographical studies", is a figure Nicholl returns to repeatedly, in both his conversation and his writing. "Aubrey is a great hero," he says, "both for his antennae and for his refusal, or his reluctance, to structure, to fix, to have the last word … He's a great character because of his curiosity, his interest in the story behind the story. Always one has the sense of an official story that one's trying to get behind. He's someone who picked that lock a hundred times over … I always get defensive about him when people talk about him being untrustworthy and chaotic – he's both of those things, but in a way it's because of much greater virtues that he is chaotic and untrustworthy."

Scrupulous himself about distinctions between fact and speculation, Nicholl takes a generous view of the apparently incredible stories of earlier writers. Antonio Pigafetta, who sailed round the world with Magellan and who seems to have been, Nicholl writes in Traces Remain, "probably for the first time on a voyage of this sort, specifically there in order to write about it", described meeting a giant in Patagonia in the 1520s: "He was so tall that the tallest of us only came up to his waist." As Nicholl observes in his essay on Pigafetta, "travel-writers often exaggerate and sometimes lie", but the encounter with the giant strikes him as "something more like a hallucination".

Nicholl himself first discovered the New World nearly 40 years ago, when he won the Daily Telegraph Young Writer of the Year award in 1972, at the age of 22, shortly after graduating from Cambridge. "Part of my prize was two tickets to Martinique," he says. He went to the Caribbean with his girlfriend, then travelled on alone to South America. While there he wrote pieces for the Andean Air Mail & Peruvian Times, including "my first and last political sketch, on election day in a little village in Colombia when the hopeful presidential candidate, López, came to visit. I look on those as my first proper pieces of on-the-road journalism. I wrote them on a typewriter in the office of the Peruvian Times in Lima."

He also wrote a "wacky, Hunter Thompsony" piece about getting sucked into playing go-between and translator in a dodgy cocaine deal at a "small, white-washed café … a couple of blocks up from the waterfront in Santa Marta, a hot, scruffy sea-port on the northern coast. The wooden sign outside read, 'EL PALACIO DE LAS FRUTAS'." He sent it to Rolling Stone, who weren't interested, but 12 years later it formed the opening chapters of The Fruit Palace (1985), his rollicking – "slightly vaudeville", he says – account of infiltrating the lower end of the Colombian cocaine trade in the mid-1980s. It's "a book I would never dare to write now", he says. It's hard to read it without thinking of the remarks he's made elsewhere about travel writers being prone to exaggeration. "I get franker about The Fruit Palace as time moves on," he says. "The basic formula was to do 80% of it, and think out, imagine, the final 20%. I don't think I'm telling any sensible reader anything they couldn't guess. They'll probably query the 80%. Go down those back streets, meet those people, but don't necessarily try to walk off a boat with a briefcase full of cocaine." Even so, the book "tells a sort of truth, not necessarily quite exactly what happened to me, but it tells a truth about what it was like. It exaggerates a bit, but that in itself is a truth about that sort of place, and about the drug, and the characters."

Between trips to Colombia he was living in London, writing short "newsy" pieces about pop music and interviewing what he calls "B-list" stars – "Bryan Ferry, I think, was my most A-list" – for Time Out, the Daily Telegraph and the British pages of Rolling Stone. "When I first started I thought: this is pretty hip and glamorous. I stopped thinking that pretty quickly … You're feeling your way to what you really do want to write about." One books editor "steered me cunningly off the druggy rock journalism" by sending him a book on Coleridge's opium habit. "And I loved writing that particular review. I'll never forget it, it was a great revelation, marrying the academic interests I'd had as a student of English literature and the more journalistic side."

In 1978 Nicholl and his wife Sally moved out of London to rural Shropshire. His first book, The Chemical Theatre (1980) – a study of 16th-century alchemy and how it related to King Lear – was published two years later. Then came his biography of the "prolific and controversial" Elizabethan pamphleteer Thomas Nashe, which was published in 1984. "He was a great character and I identified so much with him," Nicholl says. "I started writing the book in my late 20s – well, about 30, say – and he died at 33. Like Marlowe and Rimbaud, you're writing about a young man. Once they stop being young men they're gone." With Rimbaud, he adds, "I never got round to writing about him properly until I was older than he was when he died." (Somebody Else: Arthur Rimbaud in Africa was published in 1998.)

His book on Marlowe, The Reckoning, came out in 1992. It considered the playwright's death – he was stabbed in a pub in Deptford in 1593, supposedly during a fight over the bill, or "recknynge" – in the context of his likely involvement in the murky world of Elizabethan espionage. In the acknowledgments to the second edition (2002), Nicholl mentions his father Michael, who had died the previous year, and "whose interest in the spy's mentality was a source of my own". "It was always a joke of one of my friends," he says, "that my dad must have been a spy because he had that English, of that generation, George Smiley-type conventional exterior."

Nicholl, his wife and three of their four children came to Italy in 1997. "I thought I was going to write a novel," Nicholl says. "I don't know if I ever will because I discovered that I seem to need material … I could never get myself to believe in any of my pages." He's hardly short of material for his non-fiction, and being in Italy "has certainly given me a lot", he says. "Though I wrote The Lodger – you can't get more English than that – sitting out here in this house in the Lucchesia … When you're writing, you metaphorically or even actually turn your back to the window. In theory I could take a laptop out into the olive groves but I don't."

He's never been tempted to write a book about moving to Italy. "I don't like the idea that you're talking to people in the morning and in the afternoon you're turning them into a character," he says. "Doing travel writing one does tend to do that, but they're not your neighbours." Those kinds of book don't much appeal to him as a reader, either. "If I hear about another bowl of freshly picked apricots I'm going to scream," he says. So perhaps I shouldn't mention Sally's homemade persimmon bread, which we had at lunch – they've a tree in the garden – or the bag of freshly picked oranges they gave me when I left.