EL Doctorow, the award-winning novelist and academic whom Barack Obama once named as his favourite author after Shakespeare, has died in New York at the age of 84.

The author of popular American historical novels including Ragtime, Billy Bathgate and World’s Fair died on Tuesday of complications from lung cancer, his son Richard told the New York Times.

In a career spanning half a century, Doctorow published 12 novels, three volumes of short fiction and a stage play, as well as scores of political and literary essays.

It was his 1971 novel The Book of Daniel - a fictionalised account of the trial and execution of Julius and Ethel Rosenberg during the Cold War – that earned him the praise of the US president-to-be in 2008, and caused the cultural critic Fredric Jameson to label him “the epic poet of the disappearance of the American radical past”.

Obama was among the first to pay tribute early this morning, tweeting:

President Obama (@POTUS)

E.L. Doctorow was one of America's greatest novelists. His books taught me much, and he will be missed.

Born Edgar Lawrence Doctorow in the Bronx in 1931, he was named after the author Edgar Allan Poe by his second-generation Russian Jewish parents and showed early talent writing for his school magazine.

“I was a child who read everything I could get my hands on,” he said. “Eventually, I asked of a story not only what was to happen next, but how is this done? How am I made to live from words on a page? And so I became a writer.”

He studied at Kenyon College, Ohio, and Columbia University, New York, telling a Guardian interviewer: “From my undergraduate days, I’ve always been interested in the major philosophical questions that don’t seem to have an answer that everyone agrees on.”

He was drafted into the US army in Germany during the 1950s, serving as a corporal in the signal corps during the Allied occupation and telling the Paris Review: “I seem to be of a generation that has somehow missed the crucial collective experiences of our time. I was too young to understand the Depression or fight in World War II. But I was past draft age for Vietnam.”

His first novel, Welcome to Hard Times - published in 1960 when he was 28 - was inspired by a stint working as a reader for a film company. “It was making me ill, reading one lousy western after another,” he told the Guardian. “So I wrote a parody in a fit of rage, showed it to the story editor, and he said: ‘This is good, you want to make a novel out of this.’ I crossed out the title, wrote ‘Chapter One’, and went from there.”

He combined writing with working as an editor through the 60s with writers including Ian Fleming, Ayn Rand and Norman Mailer, before he left the world of publishing in 1969 to write full-time.

He went on to win awards for Ragtime (1975), World’s Fair (1985), Billy Bathgate (1989) and The March (2005). In 2014, he was honoured with the Library of Congress prize for American fiction, an award he said would help soothe his chronic “self-doubt”.

Newsweek critic Peter S. Prescott wrote in 1984: “In each of his books he experiments with the forms of fiction, working for effects that others haven’t already achieved; in each he develops a tone, a structure and a texture that he hasn’t used before. At the same time, he’s a deeply traditional writer, reworking American history, American literary archetypes, even exhausted subliterary genres. It’s an astonishing performance, really.”

Doctorow himself said of his literary style: “I like commas. I detest semi-colons – I don’t think they belong in a story. And I gave up quotation marks long ago. I found I didn’t need them, they were fly-specks on the page. If you’re doing it right,” he continued, “the reader will know who’s talking.”

He married Helen Setzer in 1954, and they had two daughters and a son. “Writing is a socially acceptable form of schizophrenia. You can get away with an awful lot,” he told the Paris Review. “One of my children once said – it was a terrible truth, too, and, of course, it had to be a young child who said this – ‘Dad is always hiding in his book.’”

Author Margaret Atwood was among those who paid tribute, saluting her “oldpal”, whom she called a “pivotal” writer and “always kind and funny”.