Not all of it, he says. Some is “absolutely fantastic”. Just a certain kind of conceptual art often favoured by serious art establishments. “I go into a gallery and I find a pair of fireman’s boots and half a dozen badly tuned television sets and a half-eaten sandwich and a big wall text telling me it’s a piece of art. Maybe it’s not. Maybe it’s a lot of rubbish.”

Done and I have met at his gallery in the Rocks, Sydney, to discuss his memoir, A Life Coloured In. Almost all the paintings on display show his signature love of fresh colours, and with just a few spontaneous smudges, blobs and stripes of paint on canvas, they capture a fizzy, incandescence in Australian postcard-fodder such as the Great Barrier Reef and Sydney Harbour Bridge. There is an entire wall dedicated to the Sydney Opera House, brightly rendered like upside-down bunches of bananas.

'Art to eat with your eyes': Ken Done on 10 of his artworks – in pictures

Read more

An adjacent room is selling lines of swimwear, hoodies, sweaters and decorative home wear, although it’s a far more modest range compared to his commercial heyday in the 80s and 90s, when a thirst for his bold prints and wearable art first catapulted him to national fame.

It is a quiet weekday morning and the only other people in the gallery are an elderly Japanese couple who wander elegantly among the paintings, talking to one another in hushed tones. Done has always been very popular in Japan – in fact, for 13 years a weekly magazine, Hanako, exclusively featured his paintings on the cover of each issue.

Done and I chat in a tiny attached studio out the back, where the light comes pouring in through a wall of windows. At 75 he looks fit and suntanned, a Platonic ideal of the Aussie bloke. He sports the kind of fine, upstanding moustache you want to salute.

Ken Done: A Life Coloured In. Photograph: HarperCollins

He doesn’t use computers so hand-wrote his memoir into notebooks and on “bits of paper and backs of other things”, 140,000 words described as barely decipherable by his personal assistant, who had the momentous task of typing them up before they were whittled down to 80,000 by his editor.

The book begins with his idyllic childhood in the country town of Maclean, New South Wales, filled with fishing expeditions, homemade iceblocks and cowboy games. After moving with his family to Sydney, by 14 he had left high school to study art at East Sydney Tech (now the National Art School). The entrance exam was purely practical. “I don’t think I’d get into art school now because it’s too much theory and it’s a lot of intellectual art speak which kind of passed me by – and I must say still passes me by,” he says. The degree led him into the swishy world of advertising, and then later into fashion and painting.

There were a few bumps in the road – he survived prostate cancer in 2011, lost millions of dollars due to poor financial advice, and contended with a few harsh critics – but otherwise the book, like the man, like his paintings, is relentlessly energetic and awash with optimism.

It reads as though he’s led a charmed life, I say. “It has been and I’ve been very fortunate,” he says. “But look, it’s a memoir; it’s not a confessional.”

Done spent half of the 1960s in London as art director of top ad agency J Walter Thompson. It was a Mad Men-esque life of big-name clients (Bacardi, the Beatles), travel and parties. “It was good, it was exciting, but on a cold, grey bloody Sunday out in Windsor park and it was raining – we couldn’t even get out of our Mini to have a picnic under the trees. So we said, ‘That’s it, we’re going back.’ ”

By then he had married long-time girlfriend Judy. They left the London winter and landed in Sydney on a hot summer morning. Done remembers being “absolutely struck by the light”. It was 1969 and the opera house was just a few years from completion. New expressways were being rolled out across the land like gleaming table runners. His father picked them up at the airport and took them home to Balmoral, where his mother had a roast lamb with pumpkin, peas, potatoes and mint sauce waiting. The garden was bursting with marigolds.

“Suddenly you’re back in Sydney – and it’s an astounding city,” Done recalls. “If you’re fortunate enough as I am to live by the harbour, you never take it for granted.”

Done continued to work in advertising but went part-time when he took up painting more seriously. His first exhibition in 1980 opened on his 40th birthday. A year or so later he would paint Sunday, one of his most celebrated works which depicts the studio view from his much loved house The Cabin, overlooking Chinamans beach in Mosman. It is an homage to the happy marriage Done has made between art and beach life. In his book he describes it as a “highly decorative and sweet painting. Like a sundae. Art to eat with your eyes.”

Not long afterwards, he opened his first gallery in the Rocks, from which, as well as his art, he and Judy began to design and sell swimwear, homewares, T-shirts and sweats. The tourists, mainly Japanese, couldn’t get enough of them, and soon the locals as well. Done’s unabashed celebration of Australiana in full riot-colour quickly went from novel to fashionable. His work was everywhere, from licensing arrangements for high-end brands such as Sheraton and Oroton, to tourist markets hocking cheap knock-offs. By the late-80s they had 15 shops and 150 staff members, wholesaling across the country.

In 1993 he won the Fashion Industries of Australia Grand award, which Done considers the pinnacle of his design career. But in the years that followed market fatigue seemed to set in, and at the same time Done was eager to refocus on his one true love: painting.

Emily and Michael’s place I (1995) by Ken Done, in a tribute to Indigenous Australian artists Emily Kame Kngwarreye and Michael Jagamara Nelson. Photograph: Ken Done

In any truly great piece of art, he says, “you need to love it when you first look at it”. You might not totally understand it but the attraction is there, he says. A really good painting is one that gives pleasure over time: “It’s like a relationship: you’re not going to get everything on the first date.”

His tendency towards the sensual is not one necessarily shared by fashionable, conceptual artists today. Done says we live in the era of “the curator, the installation”. A skip, hop and a jump away from his gallery is the Museum of Contemporary Art (MCA). Currently hosting the 20th Biennale of Sydney, it is filled with suspended coffee cups, shimmering gold curtains, handwritten notes and artfully arranged second-hand luggage cases.

Me (1992) is a self-portrait by Ken Done and was his first to enter a national gallery, the National Portrait Gallery.

What the MCA does not feature is a Ken Done painting – in fact, Done has been largely snubbed by all the country’s major state and national art establishments with the exception of the National Portrait Gallery, which has a self-portrait and his portrait of architect Glenn Murcutt, and he has been hung at the Art Gallery of NSW for his finalist entries to many of Australia’s biggest art prizes, including four years of the Archibald prize.

Despite his success in those competitions, a former curator of Australian art at the National Gallery of Australia, John McPhee, once told Fairfax Done was “not considered good enough” and said “his paintings are about charm – there’s no doubt about that. They just don’t go anywhere.” He also admitted there was a “real prejudice” against designers who attempt to cross over into “serious painting”.

It may be surprising to find that an artist so ubiquitous (what Australian family in the 1980s didn’t own a Done-designed swimsuit, coffee mug or quilt cover bearing a print of technicolour fish?) considers himself “a bit of an outsider”.

Where most successful Australian artists must constantly muscle their way into the anointed halls of museums and galleries, subject to the whims of a small circle of curators, Done’s success as a brand name paved the way to a four-decade independent career as a painter. “I didn’t ever want to seek a government grant; nobody owed me a living. I wanted to be able to survive by supporting myself and my family and the gallery with what I could do.”

The down side, of course, is having to live under the shadow of that mountain of cute koala-bearing purses and coffee mugs. “It’s like singing a popular song. You sing a song [and people think], ‘That’s the guy that sang that song.’ ‘Oh no I sing other songs as well. I actually write operas.’ But that’s OK. You can’t have one without the other.”

Two years ago he told Fairfax he’d love to represent Australia at the biennale in Venice: “I think if you’re a painter, that’s what you’d like to do. If you’re a writer, you’d like to win the Booker prize.” He can’t imagine the powers-that-be ever inviting him, but if they did he “would want to make it beautiful”. He rattles off a list of adjectives he sees as epitomising Australia: colourful, exciting, open, friendly, and “not so serious”. He catches himself on that last one. “No that’s wrong. A painting of a vase of flowers is very serious but people don’t take it seriously because they think, ‘Fuck, it’s a vase of flowers.’ ”

In 2012 he strayed from his tropical fish-filled home turf into much darker territory, with an exhibition called Attack: Japanese Midget Submarines in Sydney Harbour. Done says it was “slightly frustrating” that it was this exhibition where he should attract his best reviews because “it was about death and serious things. I would argue that a picture about the barrier reef is just as serious.”

Writing in a catalogue that accompanied the exhibition, the then-MCA curator, Glenn Barkley, wrote that although “colour, light, fashion, tourism” might be the lens through which Australia understands Done, “something more exists beneath the surface of his work”. Barkley argued he should be placed alongside the Sid Nolans and the Brett Whiteleys (who incidentally once quipped: “I’d rather take methadone than Ken Done”), and other Australian art greats who lived and painted by the harbour.

“Unlike these artists, the harbour has almost singularly sustained Done’s practice, consumed him in a way that makes him unique. Perhaps he is the great chronicler of the harbour. For those who say it’s lightweight, commercial, hedonistic, can you not reply that this is a fair reflection of the city itself?” Barkley wrote.

Personally, I never stopped loving our family’s waterproof Ken Done duffel bag, in its perky bluebird blue and squiggles of underwater life. For me it captures an Australia that had finally shaken off its colonial hangover of cultural cringe, and – with an anarchic profusion of colour and brightness – boldly parted ways from its dreary, rain-washed, buttoned-up “mother country”.

Considering the healthy trade in vintage Done items on Ebay I’d say I’m not alone. The artist says his career is back on an upswing. He has recently been commissioned by the National Portrait Gallery for an exhibition on artists and their pets. In June he will show works from a recent trip to Antarctica with the McGrath Foundation, and in October the Rockhampton Art Gallery will feature a collection of his works on the Great Barrier Reef.

He says he still has much to do. “I’ve just got to get better at what I do. That’s the drive, isn’t it?”