Saturday 21 June 2014 02.00 EDT
First published on Saturday 21 June 2014 02.00 EDT

'Foreigners! Welcome to Tehran! You may also line up here," announces a smiling airport official dressed in a long, black chador who is pointing to a newly opened immigration booth. A throng of young Dutch, Polish and German travellers pick up their rucksacks and peel off from the back of our queue and rush towards the new one.

Queues for foreign passport holders at Imam Khomeini International airport have been conspicuously short for almost a decade, but 2014 is tipped to be the biggest year for western tourism in the Islamic Republic's 35-year history. The populist and bombastic former president Mahmoud Ahmadinejad left office late last year and his replacement, Hassan Rouhani, has assumed a softer diplomatic style, lifting many barriers that kept tourists away.

Although most travellers to Iran will opt for traditional destinations such as Isfahan's stunning Naqsh-e Jahan Square and Shiraz's ancient Persepolis ruins, a minority, including me, will choose less-trampled routes. New Iran Tours offers everything from fully escorted trips with guides to hotel bookings for those travelling on a shoestring. We used them to plan a trip from the lush pastures of the north-east over the vast central desert plateau to the sultry lowlands of the Persian Gulf.

Turkmen and the north-east

In Iran the car is king, and if you only have a week or two you can save a lot of time by renting one at the airport with an international driving licence. After picking up an Iranian-assembled Kia Pride and catching a few hours' sleep in Tehran we took the Old Mashhad Road north-east, past the snow-capped Damavand volcano – the Middle East's highest peak – and into the verdant valleys of Mazandaran. The province was the birthplace of Iran's great modernising king, Reza Shah, who showered millions on Mazandaran during the interwar period in the form of mechanised farms and German-engineered railways and tunnels. We pass Veresk bridge, where it is said that, in order to allay local fears that it could not support a train, Reza Shah ordered the chief engineer and his family to stand under the bridge as the first carriages passed, with the king himself riding as a passenger.

After a quick lunch of delicious heart and liver kebabs and a jug of doogh, a popular salted and minted yogurt drink, we continue to Gorgan (land of the wolves), the capital of the wild north-eastern province of Golestan, where we stop for the night. Gorgan is the perfect base for hiking trips to dozens of nearby virgin forests, such as Nahar Khoran on the fringes of the city, or the Golestan national park, a two-hour drive east, and home to leopards, wolves and goitered gazelles. Gorgan is also a short drive to the Miankaleh peninsula, a 50-mile stretch of protected sandy beach and forest, home to several species of migratory birds and the only truly unspoiled section of Iran's Caspian Sea coast.

Around 80 miles north-east of Gorgan, along a highway flanked by men hawking watermelons or dangling questionably fresh fish off long rods, lies the farming town of Kalaleh, famed for horses, silk and saffron. From here you can rent a taxi for the day for about a million rials (£23) to take you up the long and winding roads to the Khalid Nabi cemetery, a necropolis of yonic and phallic stones. (The area has many rough roads and a taxi may be better than risking damage to your hire car.)

The tallest scarps of the hills that straddle Iran and Turkmenistan are studded with hundreds more gravestones carved into the shapes of male and female genitalia to mark the final resting place of the members of a Turkmen tribe that traversed the pastures perhaps some 400 years ago.

From these hills you can see for hundreds of miles north into the Turkmen desert and south into the floral valleys of Golestan. There is almost nothing in the way of restaurants or shops in this remote and rugged corner of Iran so be sure to take supplies if you wish to stay more than a few hours. For us this means local çörek flatbread and fresh unpasteurised goat's yoghurt from Kalaleh. After our taxi breaks down (make sure yours is half decent) we are picked up by a man in a van, who after a long but unsuccessful effort to pay him for driving 100 miles in the opposite direction from his house, dropped us outside our hotel in Gorgan.

Desert plateau

We drive due south for a day (588 miles) to Tabas. It is perhaps Iran's most remote city and where, in April 1980, the US's Operation Eagle Claw reached its fiery end during a mission to free American hostages held in Tehran by the new revolutionary government. We are there on the 24th anniversary, and posters of Ayatollah Sadeq Khalkhali inspecting the wreckage of a US helicopter adorn the streets.

Ali, a young man we meet in a tea shop, drives us to the crash site, long since cleared. We sit in the car gazing at the endless unchanging landscape as a flute version of George Michael's Careless Whisper plays from his stereo. "Khomeini said Allah sent the sand to destroy the helicopters," said Ali. "Allah protects Iran. Sometimes."

The tiny oasis town of Garmeh has grown popular with Iranian city types during their winter holidays, and the Ateshooni Guest House, nestled in groves of date and palm trees, is at its heart. The owner, a taciturn hippy named Maziyar, knows the region well and can help you get around to see the best of this desert, whether it be salt lakes, rolling sand dunes or nearby villages. Maziyar's kitchen also serves fresh homemade khoresht or ghayme stews with fluffy, fragrant rice.

Make the most of the food, as Iran does not have much of an eating-out culture; restaurants are largely limited to novelty food Iranians don't get at home much – namely kebabs. Tehran and the north-western province of Gilan are exceptions. To save time, we head to Yazd for the day before catching an evening flight south. Yazd is an ancient desert city famous for its Towers of Silence, hills carved by Iran's pre-Islamic Zoroastrian people to "bury" their dead in the sky using vultures.

Gulf and islands

A 25-minute flight from Yazd takes us to the busy (and sweltering) port town of Bandar Abbas. We head to the beach, where families are paddling in the sea and young boys take tourists for horse and camel rides along the sand. From here we take a boat into the Strait of Hormuz, the geostrategic choke point for most of the world's oil, to Qeshm.

Hengam island.

The island of Qeshm is divided into a large bone-dry section where you can visit colonial Portuguese navy fortifications, deep salt caves, Martian-like rock forms and a bizarre museum exhibiting conjoined twin goats. Then on the north end of the island, mangrove trees grow up from the sea. Local guides will take you on a boat around the strange sea forests and to visit a crocodile farm. The island's bare, white rock relief is unmistakably eerie. While driving down one stretch of the island we encounter an enormous oil platform that had apparently blown in during a storm from the neighbouring UAE. Now it rests obliquely 200 metres off shore, a sad and dormant hulk repurposed as a giant birds' nest. Local fishermen will take you there for a small fee.

The same fishermen take us to Hengam, an uninhabited island whose privacy makes it possibly the only beach in the Islamic Republic where unsegregated sunbathing might happen. In the waters off the beautiful sandy beach we spot two turtles and one half-eaten dolphin. We are taken for a lunch of fried fish, flatbread and cucumber and tomato salad, served by Bandari women wearing leather face masks, the local hijab. I'm coerced into buying a grotesque ornamental elephant made from sea shells for £6.

Tehran

We arrive for the weekend (Thursday to Friday) in Tehran and settle down for a lunch at a local aab goosht (mutton curry) haunt. Dizi (Mousa Kalantari Street 52) serves a slightly less fatty, slightly posher helping of Iran's other national dish, a lamb, tomato, white bean and potato stew. There are no menus and everyone gets the same clay pot of unmashed stew, which you grind with a pestle and eat with mint, radishes and pickled relishes. In less well-heeled venues, aab goosht is the hearty breakfast of construction workers but will send to sleep within the hour anyone who doesn't burn it off quickly with manual labour.

Tehran is a conservative city, but it is also home to some of Iranian society's most liberal strands, particularly behind the closed doors of the tree-lined streets and cul-de-sacs of its northern neighbourhoods. We stay in a cheap but pleasant hotel near the beautifully bejewelled – and publicly accessible – courtyard of the Foreign Office. Its owners, from the Azeri or "Turkish" area of Iran, sit under a fan in the foyer, playing backgammon and sipping strong, sweet coffee all day long.

Grand Bazaar, Tehran. Photograph: Alamy

Serendipity sees our visit land during an Esteghlal v Persepolis football game, one of the region's most spirited derbies, at the 80,000-seater Azadi stadium in a working-class district in the south-west of the city. Sadly, women are still denied entry – a political hot potato in Iran and one that could see the ban soon repealed.

"You are for Esteghlal?" one man asks me pointedly. I nod, hoping to guess right. "Then you are my friend." From under his chair he pulls out a little bag from which emerge several tiny glasses, saucers, a flask of tea and a silver dish containing jagged sugar cubes. He pops one between his front teeth as he sips his tea.

He explains that a hundred years ago a cleric issued a fatwa to boycott sugar because the Shah had permitted Belgium an official monopoly on Iran's sugar. Iranians duly followed the fatwa but deemed it highly inconvenient and were relieved when another mullah decreed that it was OK, religiously, to consume sugar with tea as long as it is not mixed in the glass but held in the mouth. Even now, almost all Iranians take their sugar this way.

We spend a few days eating an unhealthy amount of food, visit the gigantic Grand Bazaar and stroll around the Sacred Defence Garden Museum, which, using contemporary art and photographs, tells the history of Iran's long and bloody war against Saddam Hussein's Iraq. The conflict is silently ever-present for much of Iranian society: a million men and boys lost their lives only three decades ago and their faces are splashed across the streets of Iran's villages and cities, especially in Tehran.

Later, in Tehran's bohemian honarmandan (artists') park, we are approached by a group of students, who invite us to their "club" that night to see a basketball game, a popular sport in Iran. It turns out they are Armenian – a thriving Christian minority in Iran, which makes up an impressively large chunk of the national team. The club is a walled compound with a gym and a large courtyard, where families eat kebabs and traditional Armenian food.

Iran affords its Armenian community the right to drink discreetly and waives other prohibitions of Islamic law, such as the right to buy and sell their own bacon. A few days later we are invited to an Armenian party in what can only be described as a nightclub, albeit a naff one with a preference for Adele. There is lots of hairspray and dancing and hipsters sip on a limed moonshine known as arak, or "sweat".

Emerging in the early hours and taking a taxi uptown over the huge four-laned highways that intersect this huge metropolis, we stop at a kaleh pacheh restaurant, the only kind of place still serving food in the small hours in a town of eight million people. If aab goosht is Tehran's morning food, kaleh pacheh – an entire sheep's head, hooves and testicles boiled with garlic, lemon and cinnamon – is the food of its nocturnal denizens. I can report that it smells awful but tastes brilliant – at least it does at 4am.

Despite tensions, tourism in the Middle East is growing and, for the most part, visitors can travel safely. Daniel Adamson assesses the situation in Palestine, Israel, Lebanon, Jordan, Oman and Tunisia