Here's something that's never really worked: first-person shooters on handheld consoles. Okay, Resistance: Retribution was okay, and some of the Call of Duty and Medal of Honor titles have just about scraped by on PSP and DS. But nothing really memorable, nothing that stood up as something that added to a series.

Killzone: Mercenary is aiming to be different. It wants to exist beside its home console brethren as an equal. "The key challenge was, can we make a true Killzone game on a handheld? Is that possible?" says Hermen Hulst, co-founder of Guerrilla, the Dutch studio behind the benchmark PlayStation shooter series. "You want a console-grade experience but also you want a bespoke game designed for the platform. Mercenary is very high-paced, it's very chaotic. That's what we aimed for. I think that's working out okay!"

Set in and around the time period of Killzones 1 and 2, Mercenary follows Arran Danner, an ex-UCN solder now carrying out paid missions for the Phantom Talon Corp. Crucially, he doesn't care what side he works for as long as he gets the cash, so the campaign mode will feature missions where you're fighting alongside the glowy-eyed Helghast – the baddies of the Killzone series. From this standpoint however, Danner will apparently discover some rather uncomfortable truths about this intergalactic war and the alleged good guys, the ISA.

The game has been built around the Killzone 3 engine, and it seems few graphical concessions have been made in the change to an ostensibly less powerful platform. "We were quite surprised," says Thomas Jones, the art director at Guerrilla Cambridge where much of the work is being done on Mercenary. "The Vita screen is so sharp and the fidelity is really high. One of the mistakes we made when we started was to down-rez everything too much. We realised, actually, we're going to have to put more polys and textures back in. Obviously we had to take into account the smaller characters, making sure they stand out on the smaller screen, particularly in multiplayer. But the room is there to cram more in."

During a demo session for the game, we start out trying the multiplayer. It's a conventional set-up offering six maps and three modes. We're in Guerrilla Warfare mode, which is basically Team Deathmatch. Up to eight players can take part at once, which is definitely enough for the smaller display, but not too few to leave you wandering the locations looking for action. The demo map is Shoreline, a small network of caves and tunnels, opening out onto a cliff edge with narrow walkways and a guard tower.

There are lots of little alcoves to hide in, but really this is a circuit-based run-and-gun warzone and its fun, highly competitive stuff. Landscapes have plenty of detail, the framerate feels smooth, and firefights are super-intense, perhaps because the handheld screen makes everything feel more compact and urgent. Whatever, it works.

At the outset, players are offered a series of class set-ups – a familiar range of assault, sniper and heavy gunner roles, all customisable. During the action there are also regular care package, sorry, 'Van-Guard' drops, which supply the first player to reach them with a special weapon for a limited time. Among the armoury is the Sky Fury, a targeted air strike, and the Porcupine, a rocket launcher which locks on to targets – you can then tap the screen to take them out. My favourite though is the Mantis, a remote control drone that can sneak up on players and stab them in the neck. Lovely.

It's not the only new system that will remind gamers of other titles. Mercenary also introduces Valour cards, effectively large floating playing card icons dropped by each player when they die. The suit represents the player's class and load-outs, while the rank from two to ace represents where they sit on the leaderboards. Picking up a card earns you cash and later there are large bonuses for accruing a complete suit, or a whole pack of 52. Interestingly, the value of multiplayer victories degrades over time so the leaderboards are dynamic, giving more players the chance to be king-of-the-hill for a week or so, rather than allowing one super player to get to the top and stay there. Match stats and heat maps of recent encounters will also be available, and any cash earned in multiplayer can be used in the the campaign – and vice-versa. Guerrilla is also using the Vita Party app to allow friends to quickly find each other and set up online fights.

As you'd expect from a console loaded with input options, the controls feel good; certainly good enough for a handheld platform. The two analogue sticks give you the traditional console FPS experience, and while it takes a few matches to get used to the very fine sensitivity, you're soon using the Vita layout like a Dual Shock pounding through landscapes, checking corners, swirling to catch players sneaking up behind.

The touchscreen interaction is just right. Getting in close enough for a melee kill brings up an arrow icon on the screen: drag your finger across quickly enough and you'll perform a silent takedown. However, if your intended victim performs the same move quicker, he can parry and counter. It's a bit of a gimmick but it's solidly implemented and not too intrusive.

Far better is the use of the rear touchpad to sprint. Double tapping and then keeping a finger on the pad makes your avatar run, and its a seamless experience that frees up the button array for other stuff. The rear pad can also be used to zoom in on your sniper scope, while the tilt controls let you fine tune the aiming; it's a really neat, intuitive option.

Elsewhere, the single-player campaign apparently offers around 6-8 hours of gameplay. I tried two missions: Lightning Strike and The Package. In the former, you attack a Helghast ion cannon base and take control of the huge space guns, while the latter has you escorting the son of an important embassador to an extraction point, through waves of enemy troops. I was less convinced by these than by the multiplayer. Robbed of the majesty that comes with a 40-inch display and hi-res textures, the barren wastelands and industrial complexes of the Killzone universe look rather lifeless, and there were too many obvious choke points that had to be cleared of several enemy waves before progress could be made. Also, there was a hacking mini-game. Hacking mini-games are almost universally horrible.

There's good stuff though – the AI has been built upon from previous episodes and computer controlled enemies are now really sneaky. They'll flank you, they'll shimmy across the floor to stay out of site, they'll creep up behind you. Piers Jackson, the manager of the Cambridge studio promises a more open structure to many missions. There are glimpses of freedom in Lightning Strike, where larger environments provide a range of alternative routes; Jackson hopes this will lead to more of a sandbox feel with a much more interesting set of foes.

"One of the features we wanted in Mercenary was the ability to play stealthily, to move through the environment unseen," he explains. "So we've added on systems to the more patrol-orientated gameplay of the enemies: how the AI reacts to bullet fire, to noise in the environment, what happens if they find a corpse, and the search behaviours that come out of that. How they move round the environment to find the player... these are all extensions to the AI system."

This unpredictable behaviour adds variety and excitement where the pallid locations and formulaic structure seem to let it down. Still, this is only two missions, and we're yet to scratch the surface of Danners story; certainly The Package hints at some pretty astonishing cyberpunk city environments, with towering buildings and labyrinthine neon-drenched road ways. Plus, there could be some huge surprises to come as we get to see the Helghan side of the conflict.

Mercenary then, is already looking impressive. It's fast, it feels robust and it squishes in pretty much everything we expect from a modern shooter. I'm hoping the promise of a very different storyline means we'll get a greater diversity of locations in the campaign, but multiplayer is certainly functioning better than most other handheld shooters.

And according to Hulst, this is just the beginning. I ask about the future of the Killzone series, and whether we could perhaps see, say, a Kill Zone 4 that works across console and handheld as a pervasive experience. He doesn't bat the question away quite as quickly as I expected. "From a developer perspective and as a technologist, I think that's hugely exciting. It's also not a trivial undertaking. This game is very much designed as a standalone project, but as a future potential, I'm very open to these ideas. As long as it's done right."

But Guerrilla has always been a studio obsessed with the cold hard promise of technology. If it is possible, if it is on PlayStation, it will probably happen here.

Killzone: Mercenary is due out on Vita on 17 September in the US and 20 September in the UK.

Keith Stuart attended a press trip to Guerrilla's HQ in Amsterdam. Travel and accommodation costs were met by Sony.