It probably wasn't what the Observer's intern had in mind when she got to the office that morning: being dispatched to the canal to collect twigs, leaves and other scraps of decorative foliage for a live performance by British Sea Power. But this is a band whose career has revolved around putting people – often themselves – in unusual positions.

In the course of the past decade, the Brighton-based outfit have found themselves bare-knuckle boxing with the singer of influential krautrock band Faust and – at some other extreme – have played a human fruit-machine game with Martin Clunes (involving cardboard, tin foil and three people holding lemons). Before interviewing them previously, I was once tasked with finding them first and supplied with a time and an Ordnance Survey grid reference (I ended up lost in the Sussex countryside); on another encounter, I found myself staggering around a stage in a bear costume, nearly knocking out their viola player.

Everything is more straightforward on this occasion, foliage notwithstanding, even though the band tell me they have embarked upon a new direction with their latest album, Valhalla Dancehall; it istheir fifth if you include 2009's Man of Aran, a largely instrumental new soundtrack to a 1934 cult quasi-documentary about Irish fishermen.

"We get accused of always looking backwards," says softly spoken frontman Yan Scott Wilkinson, supping ale in a pub after their recording session. "But there's not so much about the past on this record really."

"One friend described it as being like the god Janus," says guitarist Martin Noble. "It looks into the past and the future at the same time."

Invoking Roman gods to describe your songs may be unusual, but the comparison is not unfounded. The track "Georgie Ray", for example, was inspired by a 1960s Playboy article in which a group of science fiction writers comes together to discuss a vision of the future, while "Who's in Control?" could be one of the first anti-coalition pop songs, with its talk of loving local libraries and wishing "protesting was sexy on a Saturday night".

This combination of old and new is summed up in the title itself, Valhalla Dancehall. "We wanted an international flavour in there," grins Yan. "Vikings are pretty good but they're a bit… serious. A bit violent. So we mixed it with some, er, Jamaican vibes."

Valhalla Dancehall was recorded in an old farmhouse on the edge of the Sussex Downs, where Yan lived for 18 months. It was an experience that involved an infestation of mice, transforming the garden into a nine-hole crazy-golf course and learning to brave the elements. "I didn't realise you had to order your oil three months in advance," he recalls. "And how fast it runs out. The band would turn up and be unhappy because it was so cold you could see your breath."

When they formed in 2000 – the lineup comprising Yan and his brother Neil (known as "Hamilton") on bass, Noble and drummer Matthew Wood – few people would have expected such an oddball band ever to make it so far. Signed to Rough Trade around the same time as the Strokes and the Libertines, their love of heights, hiking and herons (Noble is an avid birdwatcher) didn't exactly help them blend in.

Luckily, Rough Trade boss Geoff Travis was so smitten with the band he reportedly told them they could sell zero copies of their debut album, The Decline of British Sea Power, and still make another record with him. His faith paid off and the band have steadily accrued a small but loyal fanbase – the kind of fans who will pay to watch their favourite group play down a Cornish slate mine, atop the Great Wall of China or among dinosaur skeletons at the Natural History Museum. And those weren't even the weirdest shows...

"That was when we played in a Russian town called Mosquito," winces Noble. "It was after we'd been voted Time Out live band of the year, so we played this party that seemed to be for Russian gangsters, Time Out magazine and a bed shop! There were beds everywhere. Nobody cared about us. They had their backs turned. And in the middle of where the audience should be there was a swimming pool."

Given that the band are currently celebrating a decade in pop, it seems fitting to ask for a few of their highlights. So, who's suffered the worst injury? They rifle through some gruesome memories. There was the incident when Hamilton climbed a tree to collect decorative foliage and proceeded to remove the branch he was sitting on with a saw. And there was the occasion when new member Abi, the viola player, nearly got shoved off stage by a marauding journalist dressed in a costume (for which, apologies). But it's keyboard player Phil Sumner, another recent recruit, who has the worst grimace: "Twelve stitches, three smashed teeth… all clean off. It was a swan dive at Leeds Irish Centre."

Ouch. What's been their worst financial decision? Sumner recalls a tour of Germany that ended up costing the band £3,000.

"But the worst financial decision," muses Yan, "was calling ourselves British Sea Power, so everyone thought we were imperialist twats. When we named ourselves, Britain seemed to be moving on to a bright future. Now it's just gone back to the 1980s and it's a bit embarrassing."

So, after 10 years of trying to turn songs about celebrity clay-pigeon shooting contests into hit records, what's the closest you've come to feeling like famous, rich, successful pop stars?

This time there's no debate. It was when Sumner got a discount in B&Q after being recognised by a member of staff. He smiles: "Essentially, that's all we've ever wanted… a good discount on power tools."