A soldier motions to a helicopter in Vietnam, 1970. Photograph: Rex Features

In the summer of 1970, Karl Marlantes, a recently demobilised Vietnam veteran posted to US Marine Corps headquarters after 13 months of highly decorated active service, found himself walking some sensitive military papers across to the Capitol. He was challenged by a group of young anti-war protesters "hollering obscenities", chanting "babykiller" and waving north Vietnamese flags.

Matterhorn

by
Karl Marlantes

"I was stunned and hurt," he recalls, speaking to me during a recent visit to London. "I thought, you have no idea who I am… yes, I wanted to shoot them. Six weeks before, I was killing North Vietnamese guerrillas in combat." As his immediate rage moderated into puzzled anguish, Marlantes found himself wanting "to explain myself to those kids. I just wanted to tell my story".

So he began to work on his Vietnam novel, taking a title, "Some Desperate Glory", from a line in Wilfred Owen's "Dulce et Decorum Est". The national trauma of the war was dragging on and he intended to address something huge in the life of contemporary America. "The Vietnam war was a defining experience in the US," he says. "It made this incredible divide, even within families. The Democrats were anti-war and the Republicans supported our troops. It shaped a generation, at least, and conditioned our response to things like Iraq and Afghanistan."

By 1977, Marlantes had completed a massive, first-person narrative, full, he says, of "psychobabble" and an unmediated bitterness that he's now embarrassed to contemplate. No publisher would touch it. So he went back to a second draft, and a third…

Finally, 35 years after he first sat down at his manual typewriter – by now divorced and in his 60s – he completed the novel that's called Matterhorn, a debut that has been hailed by American critics as the definitive Vietnam novel of our times – "One of the most profound and devastating novels ever to come out of Vietnam" (New York Times).

The title is derived from the codename for a remote, mountainous military outpost, a "firebase", near the demilitarised zone (DMZ) separating North and South Vietnam and the Laos border, not unlike the notorious Hill 937, or Hamburger Hill. "Matterhorn" becomes a killing field for the young marines of Bravo Company, as they repeatedly try to secure a patch of Vietcong ground. They are led by a young second lieutenant named Waino Mellas, who has much in common with Marlantes: an Ivy League graduate from rural Oregon who adheres to the values of his childhood rather than the smart, east coast radicalism of his Princeton roommates. Mellas volunteers for the Marine Corps and, wet behind the ears, takes command of a platoon in the north-west corner of South Vietnam during the rainy season of 1969, just as Marlantes did. "All second lieutenants in history are the same," he says. "I was just a young white kid from Oregon commanding these working-class kids from the ghetto."

When Marlantes began to commit this experience to paper, the books that influenced him, he says, were the classics: "War and Peace, The Naked and the Dead and the literature of the first world war. It's amazing how that war, and writers like Sassoon, Graves and David Jones [In Parenthesis], have shaped our image and understanding of war. The first world war was the first mechanised war. Survival was more about luck than skill. Where the shells burst. And that was my experience, too."

While Marlantes was painfully translating his tour of duty into fiction throughout the 70s and 80s, supporting himself by working as an energy consultant, the US was coming to terms, creatively, with its national nightmare. The first successful account of Vietnam occurred in non-fiction, in 1977, with Michael Herr's Dispatches, a landmark volume of reportage based on Herr's visits to Khe Sanh for Esquire at roughly the same time that Marlantes was attacking his Matterhorn. Herr's achievement was to find a voice in which to describe an unimaginable apocalypse. It was described by John le Carré as "the best book I have ever read on men and war in our time".

"You know how it is," wrote Herr, describing the dead, "you want to look and you don't want to look… once, I looked at them strung from the perimeter to the treeline, most of them clumped together nearest the wire. Then I heard an M16 on full automatic, starting to go through clips, a second to fire, three to plug in a fresh clip, and I saw a man out there, doing it. Every round was like a tiny concentration of high-velocity wind, making the bodies wince and shiver."

From this kind of psyched-up non-fiction, it was a short step to the movies, the first art form to undertake the excruciating process of imagining the unimaginable. Herr contributed some of the voice-over to Francis Ford Coppola's Apocalypse Now in 1979.

Marlantes remembers that "in their surreal way" the movies of the 70s and 80s helped to focus his imagination and give it permission to roam at will across the no-man's land of America's historic defeat. "How can any modern novelist not be affected by the movies?" he says. For Marlantes, The Deer Hunter was "a fine piece of movie making", but nothing to do with his Vietnam, as he understood it. "Only Platoon came close to getting it right," he says. Instead, it was the next generation of drama, about the second world war, for example Band of Brothers, that would make the biggest impression. "Spielberg and Tom Hanks are the guys who were getting it right," he says.

The moral drive of fiction is faithfully to "get it right" through the contrivance of making it up. Ideally, the novelist must be Everyman to convey the essence of a situation in a universal language. This is a tall order when it comes to a subject that is both intrinsically unsharable (not everyone can be a GI) and innately unimaginable (few ex-soldiers want to talk or write about what they have seen and done).

At the same time, a writer needs tranquillity and perspective in which to recollect the emotion. The bigger the trauma, the longer the necessary perspective. Marlantes continued to wrestle with his magnum opus, in draft after draft, occasionally questioning his fitness for an impossible task.

Others were beginning to find their voices. Ron Kovic wrote Born on the Fourth of July during one hectic month in the mid-70s. Shortly after Herr's Dispatches, Philip Caputo published A Rumor of War, another non-fiction account. Unlike Herr, always a reporter, it was a first-hand expression of combat, a veteran's autobiography that raised the bar for getting to grips with the graveyard called Nam.

In his foreword, Caputo set out his artistic credo. A Rumor of War was not a history or a "historical accusation". It was, he said, conscripting the language of fiction, "a story about war, based on personal experience". Taking his cue from Mailer in The Armies of the Night, the author puts Lieutenant Philip Caputo centre stage in a deployment of marines to Da Nang. The narrative moves from a rational, semi-detached opening entitled "A Splendid Little War", through a section depicting the madness of war ("The Officer in Charge of the Dead") to a full-blown nightmare ("In Death's Grey Land") in which the Caputo character is accused of shooting Vietcong suspects and faces a court martial. Eventually, the charges are dropped and Caputo is reassigned to a desk job followed by an honourable discharge.

An epilogue addresses the memories awakened by the fall of Saigon in 1975. A Rumor of War was possibly the first time a former serving officer had addressed the role of the marines in Vietnam. Merlantes, also a marine, and proud if it, acknowledges that Matterhorn builds on Caputo's groundwork.

By the mid-80s, the war in Vietnam was becoming lost in the slipstream of history. All that remained of a national tragedy were the terrible craters left by the B52s and the rusting military hardware on the beach at Da Nang. The memory of the war was kept alive by veterans' rage, investigative journalism and the quest for war crimes. In Sideshow, William Shawcross exposed the role of Henry Kissinger in Nixon's secret war against Cambodia, and Born on the Fourth of July became a hit film, starring Tom Cruise.

In fiction, meanwhile, the platoon was beginning to emerge as the definitive unit of humanity in the face of battle. The leading exponent of platoon fiction was Tim O'Brien, first in Going After Cacciato (1978), and then in The Things They Carried (1998), a sequence of linked stories based on O'Brien's experiences. In "Good Form", the narrator introduces a new element into writing about Vietnam, drawing a distinction between "story truth" and "happening truth", an allusion to Daniel Defoe's famous description of the novel as "lying like truth".

O'Brien seems to be arguing that telling a story which is technically inaccurate yet truthful about the sensation of war – as opposed to baldly stating the facts of a situation – is the more honest way to report the veterans' experience, while at the same time assuaging the writer's conscience. In relation to this distinction, Marlantes says of his fictional mountain: "Everyone knows what 'Matterhorn' is. It's every hill in Vietnam. I've tried to explain what it was like. Sometimes it was so hard I would start to cry at my typewriter and say, 'I can't do it.'" Never did fiction feel less like entertainment.

O'Brien, Caputo and Kovic, among others, universalised Vietnam – whose jargon passed into the common currency of the time – as a shorthand for the madness of a jungle war. The US, meanwhile, continued slowly to come to terms with its past. The election of Bill Clinton, who had not served in the military, in 1992, was one kind of milestone on the road to national sanity. But the war would not go away. In 2000, George W Bush's alleged "draft dodging" became a campaign issue, though not a decisive one. In 2004, John Kerry attempted to capitalise on his career in the military and was unceremoniously "swiftboated" by Republican veterans.

As the US continued to make peace with its past, Marlantes continued to write and rewrite his manuscript. After Iraq, so alienated from war had the public become that some publishers to whom he showed his work advised him to cut it in half and relocate it in Afghanistan. But he refused to deviate from his course. "It was now twice the book it was in the beginning," he says. "Then, I had no compassion for any of the characters. Now, with maturity and distance, I had come to love them all."

Many books and films about Vietnam have been unable to suppress a persistent strain of fear and loathing for the place. For Marlantes, the impulse was to celebrate a noble sacrifice and to make his novel an act of homage to the fallen. There is nothing derogatory about Matterhorn. With the passage of time, too, he had found a way to deal with the unmentionable face of conflict – the inevitable racism of the frontline where whites were fighting alongside black troops. "You cannot imagine how racist the army was in the 60s," he says. "Out in the field, we were held together by fear, but once the troops were back at base the old divisions, black and white, would come back."

Truthful and painful or not, still nobody was interested in the story he was telling. "There would be times," he says, recalling his long march to publication, "when I'd say to myself, 'If you don't believe you've got the talent to do something better than everyone else, you'd have to be crazy to go on.'" The process of composition was accompanied by the nightmares of post-traumatic stress disorder. Remarkably, after more than 30 years, the novel exudes a desperate fury as Marlantes drags the reader (and Bravo Company) through firefight after firefight.

Combat is not Marlantes's deepest subject. Metaphysically, he wants to grapple with the relationship of killing to the nature of evil. In a key passage, he writes: "No, the jungle wasn't evil. It was indifferent. So, too, was the world. Evil, then, must be the negation of something man had added to the world. Ultimately, it was caring about something that made the world liable to evil. Caring. And then the caring gets torn asunder. Everybody dies, but not everybody cares. It occurred to Mellas that he could create the possibility of good or evil through caring. He could nullify the indifferent world. But in so doing he opened himself up to the pain of watching it get blown away."

It's at moments like this that Marlantes steps alongside Stephen Crane (The Red Badge of Courage), Joseph Heller (Catch-22) and even Ernest Hemingway (A Farewell to Arms). But even after 35 years, his life's work was no nearer publication.

The typescript was a beast, some 1,600 pages. No one wanted it. Vietnam was passé; first novels were a no-no; the author was too old; and so on. Spurned by agents and battered by rejection, Marlantes placed his book with a small non-profit publisher in Berkeley, California – El León Literary Arts.

His luck began to turn. Morgan Entrekin, president of Grove/Atlantic, decided Matterhorn was "the Vietnam novel of our generation" and persuaded El León to go into a commercial partnership. Entrekin also persuaded Marlantes to cut and sharpen his battered manuscript from 800 to 600 printed pages in one final edit. "Every cut hurt," says Marlantes, "but if I wanted to reach a wider audience, this was what I had to do."

Marlantes has been rewarded for his determination to tell his story: currently, Matterhorn is no 3 in the New York Times bestseller list. The commercial tide is turning towards Vietnam stories again. For instance, Tatjana Soli's forthcoming first novel The Lotus Eaters, set during the fall of Saigon, tells the story of a female war photographer who must choose between her Vietnamese lover and her journalist ex-boyfriend. The conflict has begun to join the US civil war as a national trauma that is finally sponsoring art in new and unexpected ways.