Even with a new film to sell, Guy Maddin is not your standard-issue eager-to-please director. "So many people are baffled," he says, with well-practised irony. "The movie will be crystal-clear upon your third viewing." This is Keyhole, Maddin's ninth full-length film since 1988; and against all the odds it's secured a theatrical release in the UK. Most of Maddin's work simply doesn't get to Britain, so resolutely has he followed his own path.

If you know him at all, it is probably for his ballet film Dracula: Pages from a Virgin Diary, or just possibly My Winnipeg, his heartfelt docu-essay tribute to his Canadian hometown. More energetic cineastes may remember 2003's The Saddest Music in the World, Maddin's most determined shot at the mainstream, an elaborate parody musical starring Isabella Rossellini. The truly dedicated will have sought out Tales from the Gimli Hospital, his 1988 feature debut, set in the remote Canadian settlement of Gimli, some 50 miles north of Winnipeg. All share Maddin's preoccupations: a swirl of phantasmagoric imagery and cut-up surrealism, filtered through his particular brand of cinema ancestor-worship (German silents, 50s Technicolor, big-hat noir) and occasional shafts of very daft humour. Clarity of narrative and ease of access is not high on the agenda; Maddin would appear to be the closest living counterpart we have to Jack Smith, the underground legend of Flaming Creatures renown.

In the flesh, the 56-year-old Maddin turns out to be an avuncular, energetic talker, not afraid to discuss his elaborate concepts in detail. Keyhole, it emerges, was made as a result of "a bunch of dreams I was having that have really been haunting me". "Melancholy dreams," he says, "where I revisit my past: dead relatives and homes that have meant a lot to me, particularly my childhood home. I can remember them far better in the dream than I can in waking hours."

Guy Maddin: 'Whether I want them to or not, my movies end up disjointed and ­disconnected and discontinuous.' Photograph: Jonathan Leibson/WireImage

Chief among these, it would seem, is Maddin's own father, Chas, a general manager for Winnipeg's ice hockey team who died in 1977, and who Maddin has frequently revisited since in celluloid. Keyhole mashes together noir-gangster imagery with Homer's Odyssey – "The Odyssey is a story witnessed through the eyes of a son who idolises his father as a great warrior, and I had a childish notion of a father as a strong alpha-male gangster" – and sets up a frankly bizarre and sketchy-to-the-point-of-nonexistent storyline of a hoodlum journeying from room to room in a seemingly haunted house. "New-age friends of mine say I should say goodbye to my father and wrap him up in a blanket and put him in the ground for ever. But I don't want to. Since we all live in the past and present simultaneously, and the past makes us who we are in the present, destroying it is such a cockeyed notion."

But Keyhole's tenuous relationship with narrative logic, or even moment-to-moment comprehensibility, means it's perhaps a tad on the self-indulgent side? "Whether I want them to or not, my movies end up disjointed and disconnected and discontinuous. I feel the only way I can get away with such loosey-goosiness, even for my own judge and jury, if it's genuinely coming from some place honest."

Maddin can't resist a little facetiousness. "People have been telling me for years I do myself no favours by making movies that are difficult to classify, so I went all out to make a genre piece to push it through to blockbuster domination." He cites the unexpected success of the Dracula film as a forerunner: "People had an idea of what they'd get, and they got it. People showed up for it. It was supposed to be a one-off thing for TV and ended up getting a theatrical release. The advice I had been given for decades was right: working in the genre mode really helped."

Still, Maddin wasn't about to give up his habitual idiosyncrasies. He staged "collage parties" to help generate script ideas for Keyhole: "I invited the best young up-and-coming scene-grabbing artists, in various cities. I would prime their pumps with a few words – 'electric chair', things like that – and supplied a stack of old melodrama magazines, a stack of porn, and a few kegs of bourbon. We embarked on a very peaceful and therapeutic and yet disruptive process of snipping paper into blizzards of nipples."

Maddin says he sold a few of the collages to help finance the film: "I should have kept them, on the off-chance they'd go up in value, but I needed the money pretty badly." This underlines the fact that Maddin occupies a special niche, midway between the cinema world and fine art. Money seems to be increasingly on his mind, and explains why he is cropping up so often in galleries. "I never thought about money in the past; but I decided when I turned 50 I was going to have to. I have no savings, no retirement plan, and the art world just seems wealthier. I started to be more serious about it four or five years ago, and I liked my initial sorties there."

You also get the sense that the art world is as accepting of film-makers as cinema is of artists: Maddin mentions Apichatpong Weerasethakul, but he might just as well have taken cues from Patrick Keiller or Matthew Barney. For the past year or so, Maddin has been working on a film "seance" project called Hauntings (or Spiritismes, when it fetched up at the Pompidou in Paris earlier this year). The idea is that Maddin and his cohorts would "contact" the ghost of a lost film – William Wellman's Ladies of the Mob, for example, or Mikio Naruse's The Strength of a Moustache – and then recreate/reimagine it as if under the influence of the cosmic ectoplasm. It's expensive, says Maddin, at half a million dollars a go, but he puts on a good show.

Maddin says he tried shooting some of the seances simultaneously with Keyhole, on the same set ("I built a little studio system where I deployed eight guest film-makers from various parts of my past to shoot in different rooms"); but Maddin's attempt to be the Louis B Mayer of art cinema had to be abandoned after "the demands of the two projects started hurting each other". In practice, this meant actorly rows: Maddin giggles at the memory of his Keyhole stars Jason Patric and Udo Kier storming off set after the latter "started screaming out as Kaiser Wilhelm in The Beast of Berlin in one corner while Jason was trying to achieve some serene, wrathful interiority on Keyhole in another".

Maddin remains an unrepentant avant-gardist, a film-maker committed to walking the untrodden ways, far from commercial gain and sobbing acclaim. Keyhole is no exception. "I wanted to make something viewers could let themselves go with and just listen to, like a piece of music. I don't mean listen with their ears, I mean listen with their eyes; and feel no need to understand it, but just take it in." It may not be to everyone's taste, but cinema is all the richer for his trying.

• Keyhole is released in the UK on 14 September

This article contains affiliate links, which means we may earn a small commission if a reader clicks through and makes a purchase. All our journalism is independent and is in no way influenced by any advertiser or commercial initiative. By clicking on an affiliate link, you accept that third-party cookies will be set. More information.