With the thermometer hovering around 40C (104F), we negotiated the mountain track to Hoshteva. The white stones reflected the noonday sun as we urged our horses round the last bends into the village. To the left a handsome Orthodox church, straight ahead a sign for a café bar. No prizes for guessing which we visited first. In such a remote spot, we hardly dared to hope for refrigeration as we tied our horses up under the trees. Miracle of miracles, we found it. Korça beer or Coca-Cola, glugged down on a shady veranda, have rarely tasted so good.

On this occasion, our group of British horsemen – a banker and his wife, an antiques dealer and a social worker – had the advantage over Lord Byron and his Cambridge friend, John Cam Hobhouse, pioneering long-riders in the mountains of southern Albania in 1809. In peace time, the odd couple would have made the Grand Tour through France and Italy, but the Napoleonic Wars prompted a more ambitious adventure, starting in the brothels of Lisbon, sailing across the Mediterranean to Greece and riding north to Tepelenë, home of the despotic Ali Pasha, who proved an unexpectedly generous host.

This agenda was no hardship for the bisexual Byron: at the age of 23, he welcomed the Levant’s potential for passions that were illegal back home. His journey inspired “Childe Harold’s Pilgrimage”, the poem that created the Byronic hero, intemperate, self-indulgent and irresistible. While he worked on his immortality, poor Hobhouse, described as “short, plain, furtive and unhygienic”, charted their progress in more pedestrian terms.

“A great deal depends on your choice of dragoman because he is your managing man,” he wrote in his diaries. “He must procure your lodging, food, horses and all conveniences.” We had our own dragoman, a thoroughly Ottoman Mr Fixit, in Auron Tere, resident in Michigan, but born into the upper echelons of Albanian society in Gjirokastër, a fortified town that made it onto the Unesco World Heritage roll of honour in 2008.

Albania, its independence guaranteed by the 1913 Treaty of London, claims that it’s the only country in the world surrounded by its own rightful territory. If the people are to be believed, it has been systematically land grabbed over the century of its existence by its neighbours, currently Montenegro, Kosovo, Macedonia and Greece. What remains is a country with a population of around three million, 70 per cent Muslim, the rest Orthodox Christian and Catholic. It has the last barely touched Mediterranean beaches in Europe and a disproportionate number of car washes. It’s no accident that immigrant Albanians dominate that particular roadside trade in Britain.

In the mid 20th century, Enver Hoxha, another son of the Gjirokastër elite, gave it a very bad press. The Stalinist dictator ruled with an iron fist from 1944 until his death in 1985. Five years later, Albania established the fragile democracy that exists today. Although Hoxha’s brutal legacy persists in authoritarian politics and contempt for the rule of law, the country now welcomes visitors warmly, just as it did in Byron’s day.

After we’d visited Gjirokastër’s medieval castle and Hoxha’s luxurious birthplace, Auron drove us south towards Greece, veering off road into a meadow where we found our fellow travellers, Fuat and Gus, supervising a row of sturdy horses. Briskly Auron unloaded English-style saddles from his Land Cruiser, placed them on small, hairy backs and allocated steeds to riders. Nelson, tiny, shiny and fiery, was reserved for Dom Mocchi, our effervescent Italian leader. Mandela or the Admiral? Auron shrugged: either way, it was the perfect name for a black one-eyed horse.

Riding holidays often settle into a routine, mornings and afternoons in the saddle, a picnic lunch, an evening campsite and dinner under the stars, but Albania is a land of the unexpected. On day one, we rode down an avenue of cypress trees to the Bektashi monastery at Melan to meet our first dervish. Not the whirling kind of popular imagination, but a Sufi mystic who had embraced the Bektashi celibate life in 2000 at the age of 15.

Now a bearded adult dressed in silky white robes belted in green and gold, Myrteza lives with his mother in an outsize monastery overlooking the Drino river valley. The dramatic mountainscape is interrupted by a metal tower at the end of the terrace. In the modern era, the archaic dervish lifestyle, its emphasis on study and prayer, is more viable when a mobile phone company picks up the tab.

Unfortunately Myrteza’s studies don’t include foreign languages but he welcomed the break provided by passing strangers with smiling enthusiasm. While he chain- smoked, his mother served tea and raki, not the aniseed-based Turkish kind, but a fruit schnapps resembling grappa. Other Muslims consider Bektashi Sufism heretical. During this relaxed interlude, it was easy to see why.

Back on Nelson in an elated mood, Dom went ever higher up the hillside, bushwhacking an arduous trail through rocks and shrubs rather than sticking to the path. As an Italian, he shared linguistic links, a zest for life and an indomitable spirit of adventure with our Albanian hosts, so riding in his wake was never boring.

In Libohovë, a large village dominated by the remains of Ottoman fortifications, we ate under the 250-year-old tree that sheltered the terrace of the only restaurant. Just like Byron? Who knows, but it would have been large enough when he passed this way.

History insists that he rode over our next day’s destination, the ancient site of Antigonea, though he wouldn’t have known it because excavations didn’t start until the Sixties. The ruins of a major Greek trading city founded by King Pyrrhus of Epirus in 295BC and burned by the Romans a century later are the focus of a well presented National Archaeological Park. As a good dragoman should, Auron had acquired a prized pass for a group sleepover. While we rode, he’d been out foraging. Now the Land Cruiser returned laden with warm spit-roast chickens, savoury flaky pastries (burek), fresh bread, fruit and cold drinks. Let the feasting begin.

The climax of our ride was a three-day camping trip in the Bureto Mountains, starting in Erind and ending in the Kelcyra Gorge. Unexpectedly, Albania comes second to Finland in Europe’s per capita hydroelectric tables and much of that power is generated in this rainy, wooded area. The villages, built into the hillsides with walls and roofs hewn out of rock, are almost invisible until you ride into them. In most of them, we found Orthodox churches, often in poor repair but with icons lit by naked bulbs glowing in dark interiors.

Many of the younger residents have headed south to look for work, but those who remain opened their doors with smiles of delight. As the sun went down on a long day in the saddle, an 80-year-old granny, one of five year-round residents in a remote village, welcomed us onto her vine-shaded terrace and heated the coffee pot. Her son, on holiday from Athens, broke out a bottle of raki, a merry interlude that set us up for our first night in the open.

Now that we were adrift from civilisation, our camp set up by Fuat and Gus in a pasture high above the Vlose Valley, Auron was on the case full time. Pack horses transported tents, luggage and provisions, but our dragoman sourced water from pure springs, enormous circular loaves from village ovens, cheese and yogurt from mountain herds. Meatballs appeared as if by magic accompanied by tomatoes and cucumbers, a default Albanian meal that has a lot going for it.

Afterwards we fell asleep to the sound of horses chomping rich grass and wearing tinkling bells so they could be found in the morning. No lie-in, though: a dawn goat invasion was guaranteed, followed by a site inspection by curious villagers talking loudly in a language that sounds like fighting talk even when it isn’t.

All too soon it was time to descend from the uplands and rejoin Byron in a landscape that reminded him of his childhood home near Aberdeen. Just as he did, we rode along the turbulent Vlose, a startling turquoise green as it plunged through the Kelcyra Gorge. “Land of Albania. Let me bend mine eyes on thee, thou rugged nurse of savage men,” he wrote in “Childe Harold’s Pilgrimage” of the people that crystallised his innate revolutionary zeal. In Athens, his next destination, it would find an outlet in the struggle for Greek independence, a cause that dominated the rest of his life.

What of lesser men? We embraced the cool of a pretty riverside hotel as we tucked into trout newly caught from the river on a terrace bright with flowers. Nothing so poetic as the maestro, I’m afraid, but a fine end to a journey of discovery in a rewardingly uncharted land.

Albania Horse Trek: In the Footsteps of Lord Byron, from May 24 to June 1, 2013, costs £1,495, not including return international flight, with Wild Frontiers (020 7736 3968; wildfrontiers.co.uk).