Poor Dave Van Ronk. He was in the right place – the Greenwich Village coffee-house scene – at the right time, doing all the right things, singing the right songs to the right people. But he just didn't have the magic. And he didn't have the luck, either.

Sometime in the 1950s, when he was a young man trying to become a folk singer, he had learned a traditional song called "House of the Rising Sun" from a pre-war field recording on which it was sung without accompaniment by a Kentucky miner's teenage daughter. Van Ronk changed it around a bit, keeping the tune and most of the words, but adding a distinctive chord sequence that made an already plaintive lament even more arresting. As his reputation grew, it became the highlight of his stage act. "It was Dave's song," a witness to the era recalled. "He owned it."

One night in 1962, a younger singer approached him with a request. "Hey, Dave," Bob Dylan said, "would it be OK for me to record your version of 'House of the Rising Sun'?" Dylan had just signed a recording contract with a major company, while Van Ronk was preparing his own debut for a small folk label.

"Jeez, Bobby," he replied, "I'm going into the studio to do that myself in a few weeks. Can't it wait until your next album?"

"Uh-oh," Dylan said, immediately arousing Van Ronk's suspicions.

"What exactly do you mean, 'Uh-oh'?"

"Well," an embarrassed Dylan said, "I've already recorded it."

So he had, and the song became a key track on his first album, which reached only a limited audience that year but helped lay the foundations of his career. "House of the Rising Sun" was its most immediately striking track; on both sides of the Atlantic, aspiring folk singers even younger than Dylan avidly learned the song from his recording.

For Van Ronk there was no credit, and there were certainly no royalties. There was only the measure of grim satisfaction that came three years later when the Animals copied Dylan's recording and took "House of the Rising Sun" to No 1 in the US and UK pop charts, crediting the arrangement to their organist, Alan Price. Now Dylan himself could no longer perform the song, because people would assume he had stolen it from the British group.

Van Ronk told the story in his autobiography, The Mayor of MacDougal Street. Published in 2005, three years after his death at the age of 65, it describes the glory days of the Village, and is full of amusing character studies and pungent opinions. It also forms the platform on which the film directors Joel and Ethan Coen constructed their latest film, Inside Llewyn Davis, a portrait of a New York singer during the period in which folk was taking over from jazz as the music of the hip intelligentsia.

The Coens' focus is sharp. The time is early 1961 and the streets of the Village are covered with snow. Folk singers who can't afford winter coats perform in starkly furnished coffee houses to audiences of college students swiftly discovering alternatives to conventional Eisenhower-era ideas of progress towards adulthood. This is a world in which idealism battles against careerism, and, in the case of Llewyn Davis, a veneer of one barely conceals a core of the other.

Most of those seeing the film's title for the first time will think of Dylan, who branded his new identity with a Welsh poet's name. Those scenes set in the snow-covered Village streets are a direct translation of the cover photograph from The Freewheelin' Bob Dylan, the 1963 album with which the singer's career began to take off. And, rather naughtily, the cinema trailer is set to a soundtrack of Dylan's "Farewell".

But Llewyn Davis is not Bob Dylan. He is not Dave Van Ronk, either, despite the borrowing of mood, texture and many salient incidents from The Mayor of MacDougal Street. But he does occupy a place that history reserved for Van Ronk, as the man who preceded Bob Dylan and was destined to find himself completely overshadowed.

At 20, Dylan was only four years younger than Van Ronk when he arrived in New York in January 1961, but it was a significant four years: one was a pre-war baby, the other a baby boomer. Different sensibilities, different expectations. Dylan had come from Minnesota and was intent on escaping the comfortable existence offered by a stable, relatively prosperous family and remaking his life, starting with the invention of a colourful backstory for the benefit of his new friends and new audiences.

Van Ronk had actually lived the sort of life Dylan made up. Born in Brooklyn to parents of Swedish origin who separated soon after his birth, he never knew his father and "never felt a twitch of curiosity about him". His mother, a stenographer, shared the job of bringing him up with a succession of aunts, one of whom had been a rum-runner for the gangster Legs Diamond and managed a speakeasy during prohibition. "She loved jazz," Van Ronk writes, "and in that house the radio was always playing: Duke Ellington, Louis Armstrong, Benny Goodman, Count Basie …"

Later in childhood he would spend time with a grandfather who had been a hotel pianist and could play "The Maple Leaf Rag". His Brooklyn-Irish grandmother was a great singer, and taught him songs that would become staples of the folk repertoire. (It was to one of her favourites, "The Chimes of Trinity", that Dylan would pay particular attention: "He made me sing it for him a few times until he had the gist of it, then reworked it into 'Chimes of Freedom'. Her version was better.")

By the time Van Ronk got to high school he had decided to become a musician, and increasingly regular visits from the truant officer put an end to his formal education at the age of 15. By then he had acquired a ukulele and learned his first blues, "St James Infirmary", from a copy of The Fireside Book of Folk Songs kept on another aunt's piano. The first guitar came in a schoolyard swap, and Van Ronk devoted himself to mastering the instrument, a task made more demanding by having been forced in childhood to "correct" his natural left-handedness. Lessons came from a neighbour who had played guitar with Jean Goldkette's band, and had known Bix Beiderbecke.

In his teens when he made his first visit to Greenwich Village, Van Ronk was expecting to find "half-timbered Tudor cottages with mullioned windows and thatched roofs, inhabited by bearded, bomb-throwing anarchists, poets, painters and nymphomanics". He was shocked, as he emerged from the West 4th Street subway station, to discover something that looked disconcertingly like Brooklyn.

Nevertheless, the Village became his natural home, a place of kindred musical, political and social spirits where he could drop by the White Horse Tavern to hear Dylan Thomas read his poems, spend afternoons at the Five Spot listening to Thelonious Monk rehearsing with John Coltrane, or make friends with surviving exponents of early jazz, whose wisdom he absorbed as he began playing with Dixieland bands in and around New York. In between times there were spells as a merchant seaman and road trips across America.

The switch from jazz hound to folknik came after Van Ronk had acquired the habit of spending Sunday afternoons in Washington Square Park, listening to kids with banjos and guitars singing songs they had learned at "progressive" summer camps. One of them taught him the art of fingerpicking. For Van Ronk, it was "a change that defied the general rule that things evolve from the simple to the more complex.

"In this case I made a move that was technically retrogressive, but it was about the only thing I could do to survive … much as I loved traditional jazz, I was sick to death of performing the music of King Oliver and Jelly Roll Morton for drunken undergraduates who wanted us to put on funny hats and sing 'Yes, Sir, That's My Baby'."

But elements of his background clung to his new identity, and even after he embraced the folk repertoire – soaking up the material collected on Harry Smith's Anthology of American Folk Music, sitting at the feet of Mississippi John Hurt and the Rev Gary Davis – his own performances featured a broader swath of material than most of his contemporaries could muster, played with a greater musical sophistication. "The joke in the early 1960s," he recalled, "was that I was the only folk singer in New York who knew how to play a diminished chord." This turned into a disadvantage when Dylan appeared, with an approach so tightly focused that it made the older man's eclecticism sound dated.

The folk world also suited Van Ronk's desire for political engagement. He joined the Trotskyite Young Socialist League and the International Workers of the World, and his apartment on Waverly Place became, as Suze Rotolo, then Dylan's girlfriend, recorded in her own memoir, A Freewheelin' Time, "the living room of a new generation of bohemians".

Dave Van Ronk at the Gaslight in 1964. Photograph: Getty

Other Village haunts of the folk crowd included the Cafe Bizarre, the Kettle of Fish, Gerde's Folk City, Izzy Young's Folklore Centre and the Gaslight, a coffee house in a Bleecker Street basement where the musicians – including Van Ronk and Dylan – were not paid but passed a collecting basket around the audience. In Rotolo's words, Van Ronk became its "gatekeeper": it was his job "to have the tourists stay in their seats for a few sets on a quiet night, or hustle them in and out quickly in order to take in as much money as possible on weekends." The Coens create a somewhat sanitised version of the Gaslight's ambience, with its small audience of rapt young listeners and highly variable cast of performers. ("Even in the dark," Van Ronk recalled, "the Gaslight was pretty horrible.")

As Llewyn Davis, a regular at the Coens' version of the club, Oscar Isaac sings, plays and wears a beard and battered clothes with commendable authenticity, blending aspects of several personalities, including Dylan's selfishness and ambition and Van Ronk's fate as a man who would never quite make it. An unsuccessful audition for a Chicago club owner is taken directly from Van Ronk's encounter with Albert Grossman, later to become Dylan's manager. The book is also the source of an amusing scene in which Davis tries to prise royalties from the tight-fisted owner of the small label on which his album has appeared.

A year or so after Dylan had "gone electric" and achieved pop success, Van Ronk briefly tried to follow suit with a band called the Hudson Dusters. No luck there, either. But he continued to perform and record, giving guitar lessons and encouragement to younger performers, until cancer surgery was followed by a fatal heart attack.

Unlike Dylan, Van Ronk never backed away from the political beliefs that he had espoused during the years when protest songs were spilling out of every Village coffee house. Interviewed for No Direction Home, Martin Scorsese's filmed portrait of Dylan's early years, he said of the man who usurped his position as the Village's leading troubadour: "We thought of him as hopelessly politically naive – but in retrospect he might have been more sophisticated than we were."

The scenes of Davis performing – and a fleeting glimpse of a younger rival – are a reminder of how that sophistication, or whatever it was, enabled Dylan to rise above and so swiftly distance himself from a parochial scene, his youthful intensity transforming traditional material in a way that Van Ronk – however impeccable his intentions and the quality of his performance – could never begin to match. Inside Llewyn Davis is not one man's story, but it proceeds from the pain of that historic disappointment.

Interview: Inside Llewyn Davis - a melancholy movie about the early 60s New York folk scene - is another tale of failure from two of the world's most feted auteurs. Why are the Coens so drawn to losers and sceptics?