If you hear an American accent on the Channel Island of Guernsey this summer, the chances are it will belong to someone who has read a certain novel that reading groups in the United States have gone crazy for in the past couple of years. Copies of this novel are piled high in the Press bookshop in Guernsey's capital, St Peter Port. Next to them are branded mugs, bookmarks and fridge magnets.

"We sell a lot," the assistant says. "They come off the cruise ships and head straight for it. I had to disappoint an American lady last year. She wanted to meet the families mentioned in it. I had to explain that it's made up …"

The novel in question, The Guernsey Literary and Potato Peel Pie Society by Mary Ann Shaffer and Annie Barrows, tells the story, in epistolary form, of the German Occupation of Guernsey from 1940 to 1945, with a love story thrown in. It is not to everyone's taste – the bookshop assistant called it "chick lit", and I for one found it smug and twee – but it has sold in millions and had an undeniable "Captain Corelli" effect on the island itself.

It's probably fair to say that the image most British people have of this speck of granite lying off the Brittany coast is of a perennially dull backwater with relaxed attitudes to taxation. In fact, its varied topography – wild sea cliffs, soft rural interior, sandy beaches – Neolithic-to-Nazi history and fine, fresh food were a revelation to this first-time visitor. So any book that draws visitors to Guernsey to correct this misapprehension has to be a good thing.

Still, I am irritated by all the Potato Peel flim-flam because a far, far better novel about Guernsey and the Occupation has been knocking around for 30 years now, with nary a peep from the outside world, despite being lauded by the likes of John Fowles and William Golding and praised by the New York Times as "one of the best novels of our time".

This is the reason I am here – to trace the locations and events not of The Guernsey Literary and Potato Peel Pie Society but of a novel called The Book of Ebenezer Le Page. Written in local patois by a Guernsey man called G B Edwards, it is the fictional memoir of a hilarious curmudgeon, the eponymous Ebenezer, who looks back at his life on the island across most of the 20th century. The Occupation is very much in there, as is the post-war development of tourism.

"Guernsey is a factory for the manufacture of tourists now," grumps Ebenezer: " … when I go to Town [St Peter Port] nowadays and see all the visitors about, it is as much like a bad dream as when the Germans was [sic] here."

My guide, Gill Girard, is a fan. "Ebenezer is very easy to associate with," she says. "You can hear people you know speaking the way it's written. It's lovely." She laughs. "All those characters are completely feasible."

Edwards's novel – published in 1981, five years after his death – makes a rewarding alternative template for seeing the island. Ebenezer calls Guernsey "this whore of an island, this island in the sun, this island in God's sea". Gill likens it to a wedge of cheese, sloping from 300ft-high cliffs in the south to the beaches in the flat west and north.

From the airport she drives us south to those rind-like sea cliffs, a journey of about five minutes (everything takes about five minutes on this island of 24 square miles unless you get lost – easy to do in the Lilliputian lanes – in which case just keep going till you stumble on your destination by accident, and look at your watch, and see you have taken an interminable quarter of an hour).

The headland of Le Gouffre is vivid and, on this sunny late April morning, profuse with sea campion, sorrel, wild garlic, blackthorn blossom, yellow gorse and tiny purple loose-flowered orchids. Far below, a lone fisherman bobs in his open boat on the turquoise waters – a toiler of the sea, to borrow the title of the novel in which Victor Hugo wrote about this very spot.

Hugo is the literary figure most closely associated with the island. He was a political exile here for 15 years in the middle of the 19th century, in which time he wrote, among other works, Les Misérables and the Guernsey-set Les Travailleurs de la Mer. His statue stands in Candie Gardens in St Peter Port, "with his coat-tails flying in the wind", as Ebenezer notes, and his house up on the hill, Hauteville, is simply the most astonishing and inspiring writer's house you will ever visit.

This is where, in his own words, Hugo lived "as though perched on the pinnacle of a rock", writing in a lookout on the roof. Hauteville's secret doors, dark carved wood, chinoiserie, mirrors, and constant play of light and dark make it feel more like being inside a fertile imagination than a house.

But the brilliance of Hugo and his Hauteville are well documented. I am intent on righting the wrongful neglect of G B Edwards' masterful creation, Ebenezer Le Page. And so Gill drives us west, past pink and blue granite cottages with subtropical gardens, and into the pages of the old man's memoir.

Ebenezer's great love, Liza Quéripel, lives in a house at Rocquaine Bay, in the far south-west, a vast sweep of beach, rock pools and vraic (seaweed) where a scattering of mothers and babies paddle in the shallows and gaze ruminatively out to sea. From here we follow the coast road north to Lihou Island, where Ebenezer and his friend Jim are stranded all night when they miscalculate the speed of the inrushing tide.

At 33 feet, the tidal range on Guernsey is one of the biggest in the world, and visitors to the nature reserve and ruined 12th-century priory on Lihou need to study the notice of tide times next to the causeway so as not to suffer the same fate as Ebenezer and Jim.

The wide bays continue north-east in wide daisy chains of sand and rock. Ebenezer himself lives on the Chouet peninsula in the far north-west, with "only the sea and the rocks to look at". Here he also presides over "prehistorical remains".

The best of these Neolithic sites, not named in the book, is Le Déhus Dolmen, a 4,000-year-old passage grave that Gill and I walk inside. She flicks a switch to illuminate the underside of one of the capstones. The figure of a warrior is carved there, bearded, armed with a bow and arrows. It's an amazing sight, but warriors in jackboots is what Guernsey is still chiefly known for; you can't escape the Occupation. The coastline we have just driven up is disfigured by concrete towers and bunkers – the single most traumatic and dramatic event in the island's history is still commemorated and exploited in equal proportions.

This year is the 65th anniversary of the Liberation and the week after my visit, thousands of local people with direct family experience of those dark days paraded through St Peter Port to mark the occasion.

"Just to hear someone talk about Liberation Day – it's so moving," says Gill, whose family lived through the deprivation and terror. "Word got around that it was going to happen. Town was absolutely packed. It must have been amazing."

At the same time, tourists from all over the world continue to crawl over the grim concrete archaeology that the Germans left behind. As Ebenezer remarks drily, "the visitors who come over to Guernsey nowadays know more about the German Occupation than I do. They have read the books." And those books now include The Guernsey Literary and Potato Peel Pie Society.

I finish my tour of Guernsey in the German Military Underground Hospital, a vast network of subterranean tunnels and rooms stretching for more than a mile beneath the soft navel of Guernsey's interior. Who ever thought this dreadful, dank hole would be a good place for soldiers to recuperate? And what of the anonymous slave workers who hewed it from the rock? The underground hospital is testament on a grand scale to both the fanaticism and stupidity of the Nazi project.

It is owned by two troglodytic brothers, Philip and Nigel Browning, who have followed the advice of the English marketing man in The Book of Ebenezer Le Page. "There are hundreds of other holiday resorts with pretty rocks and bays," this smarm-bag tells Ebenezer at the end of the war. "The one advantage you have over the others is the German Occupation. Fence in every fortification and charge for admission…"

Philip is a canny figure with bushy sideburns, permanently hatted and coated for mid-November because the temperature in the tunnels is a steady 44.6F (7C) year round. Plenty of "Potato Peel people" visit the underground hospital, he confirms.

"Last year there was a journalist from the Boston Globe with one group and she interviewed me," he says, his voice echoing eerily through his concrete Valhalla. "I'm supposed to be on their website, but I don't have a computer." And neither, I'm sure, would Ebenezer Le Page.

Getting there

Nigel Richardson flew with Aurigny (01481 822886; www.aurigny.com), which operates from several British airports including Gatwick and Manchester; fares from £33 each way. He stayed at the Old Government House Hotel (01481 724921; www.theoghhotel.com) in St Peter Port: double b & b from £150 per night. The hotel is offering three-night "Potato Peel" breaks from October 8 to 11 from £395 per person.

Where to eat

Guernsey likes to point out that there are no fast-food outlets on the island. This is no idle boast because, as a result, even snack food is consistently fresh and excellent.

For dinner either of the two restaurants in the Old Government House Hotel (see above), the formal Governor's Restaurant or the light and airy Brasserie with a walled garden; Le Nautique (01481 721714; www.lenautiquerestaurant.co.uk) also in St Peter Port for fish and seafood.

The German Military Underground Hospital at La Vassalerie (01481 239100) is open April-October 10am-12pm, 2pm-4pm. Admission £3.50, children £1.

Reading

The Book of Ebenezer Le Page is out of print in Britain but published by NYRB in the US. Copies are sold in the Press bookshop on Smith St in St Peter Port for £10.99 and on Amazon.co.uk for £7.45. The bookshop also sells an excellent if pricey (£25) Ebenezer map of the island.