Kate Tempest likes the darkness before the dawn. Let Them Eat Chaos, the Londoner’s last album of poetic, narrative hip-hop, takes place at 4.18am, when insomnia gnaws at her characters’ brains.

When Tempest is awake at that hour, she finds it peaceful. “There is something really magic about the couple of hours before dawn,” she says. “You’re recharged, the day before is gone, but there are no requirements. You don’t belong to anybody, to anything. That time lends itself to lyricism because of the repetitive nature of insomniac thoughts.”

Tempest obviously enjoys the chance to write without fear of interruption. In the past six years, the 31-year-old has produced a novel, two albums, a play, a volume of poetry and a genre-busting spoken-word theatre piece, Brand New Ancients, which won the Ted Hughes Poetry Award. She’s a one-woman bridge between artforms and audiences that are too often Balkanised: a poet for people who don’t read poetry, a rapper for people who don’t listen to hip-hop, and more. Whether on stage or on the page, her language hits like lightning. It illuminates and it burns.

If you’ve spent a decade playing to 20 people and driven five hours to get there, you take nothing for granted

I suspect she also cherishes the solitude. She clearly prefers writing to talking. When she discusses the creative process, she finds an exuberant rhythm, bobbing in her seat and offering a glimpse of the warrior spirit she projects on stage. More often, she is still and withdrawn, staring down questions with big, blue, suspicious eyes, as if every inquiry were a potential trap.

Tempest is always writing or thinking about writing – she’s just finished a reworking of Sophocles’ Philoctetes, and has another album and poetry collection on the go – so she turns down most commitments, but she has agreed to be the guest director of this year’s Brighton Festival. Her programme, labelled Everyday Epic, includes poets, playwrights, authors, dancers, soundtrack composer Mica Levi and pop-up shows by a “storytelling army” of Brightonians.

The line-up reflects three of Tempest’s obsessions: the dialogue between artforms, the idea of artists as “cultivators of empathy”, and the life of cities. Her Mercury-shortlisted debut album Everybody Down charted a dramatic year in the life of three young Londoners. Her first novel The Bricks That Built the Houses (“an exhilarating, exhausting, terrifying process”) fleshed out the characters over 400 pages. Let Them Eat Chaos is a very different urban saga, taking place on a single London street on a single night. Her seven narrators, each one anxious and alienated, are brought together in the street by a thunderstorm and Tempest’s final, desperate plea: “Wake up and love more.”

When I meet Tempest in a Hackney studio, she has just returned from a tour of the States, where she performed the entire album front to back: no old songs, no encore, no unnecessary chat.

“The bits in between the songs can jolt you out of it,” she says. Staying within the songs is “much safer for everyone”. Despite the album’s visceral specificity – all London life is here – even festival-goers in Boise, Idaho, got the message. Tempest paraphrases James Joyce to me: “If I can get to the heart of Dublin I can get to the heart of the world.”

‘The thing that drives you might be painful, but the minute it becomes a creative act, it becomes about love’: onstage at a festival in the Netherlands. Photograph: Andrew Benge/Getty Images

Neither London nor the world come out of Let Them Eat Chaos looking good. For all its compassion, it’s a thoroughly damning indictment of modern life from root to branch. “Carcinogenic, diabetic, asthmatic, epileptic, post-traumatic, bipolar and disaffected,” she raps on the punitive Europe is Lost. “Atomised, thinking we’re engaged when we’re pacified/ Staring at the screen so we don’t have to see the planet die.” Britain, she adds, is “the land where nobody gives a fuck”.

When the album came out last October, many listeners assumed Europe is Lost referred to Brexit, but she’d written these songs months earlier.

“You might think it’s about that and it probably is, but it’s not a response to that,” she says. “It’s a response to some of the things that gave rise to that. This particular moment in history has not just fallen on us out of nowhere and if you feel this is a particularly dread time, then you’re late. You weren’t looking.”

I wonder where, in the bleak, broken terrain of Let Them Eat Chaos, she locates a glimmer of optimism?

“The whole thing is an exercise in optimism, because it’s a creative act,” she says. “The thing that drives you might be painful, but the minute it becomes a creative act, it becomes about love. It’s such a beautiful and generous practice to be able to write or perform. It’s just pure love.”

Talking to Tempest, it strikes me that the same intensity and sensitivity that makes her writing so powerful can make other parts of her job painfully difficult. She is fascinated by other people’s lives, but she bristles when other people are interested in hers.

‘I have something to prove. I don’t know where that comes from’: performing at Glastonbury in 2015. Photograph: Alamy Stock Photo

I first met Tempest three years ago, after her Mercury Prize nomination. She was shellshocked by the influx of attention and understandably guarded, but also friendly, approachable and willing to talk about how she got to that point. It’s an interesting story. Kate Calvert grew up in Brockley, southeast London, the youngest of five. Her mother was a teacher, her father worked in construction then became a lawyer.

It was a noisy, vibrant household, full of voices. Her love of language and storytelling led her to hip-hop (Slick Rick, Wu-Tang Clan) and poetry (Yeats, Blake). In her poetry collection Hold Your Own, she describes starting to rap in public at the age of 16 as an ecstatic liberation from an anxious adolescence.

She rapped anywhere she could: house parties, record shops, raves, demonstrations. Breaking into hip-hop was hard, but the first time she performed at a poetry slam, in 2006, she won and found another outlet. She studied at the Brit School (music) and then Goldsmiths (politics and English literature). Her audience grew as her art evolved, but it took a while. “If you’ve spent a decade playing to 20 people in a room that you’ve driven five hours to get to, then you take no moment for granted,” she says. “When eventually you look out and there’s 1,000 people, you know what that’s worth.”

We’re in a terrible situation. I don’t think we are prepared for what the next few years might bring

Today, though, the past is a foreign country with extremely strict border controls. Before the interview I was informed she wouldn’t discuss anything personal – not just her relationships (fair enough), but also her friends, family and childhood. It turns out that’s not all. Asking her questions is like moving a hoop along an electrified wire, knowing the slightest wobble will cause an alarming vibration. She’ll start saying something interesting, then cut it dead because it might sound like “platitudes”, or because she can “feel the headline that could be taken from it”. Even simple questions about how she thinks and feels are shot down with a doleful stare.

It’s exhausting and perplexing. If she made abstract electronic music, then this firewall between her life and her work might make sense, but her writing is so electric with insight and experience, so keenly attuned to the grain of everyday lives, that it’s impossible to remove her from the equation.

I ask her why there is so much that she won’t talk about. This is a bad idea. She looks wounded, betrayed, shrinking in her seat. It’s agonising. “I don’t trust journalists,” she says quietly. “I don’t trust this.” She indicates my dictaphone. “You and I are not just two human beings having a conversation. Your job is to rewrite, edit, package. This is not a safe place.” She sighs. “The fact I’m even talking about this makes it a big issue.”

I tell her I didn’t expect it to be a big issue. I thought she’d say something short about the importance of privacy and we’d move on.

“But that’s not the kind of person I am,” she says, exasperated. “If I’m going to engage with somebody I engage with all of myself, which is why it’s dangerous. Do you understand? I don’t exist on the surface.”

It’s true. She doesn’t have a trivial bone in her body. She distrusts acclaim and anything that whiffs of marketing or compromise, which is why she found the Mercury nomination so discomfiting. “It was a pretty mental time,” she grimaces. “I wanted to get to a place where my commitment to my craft had been proved. I have often felt I have something to prove. I don’t know where that comes from. Maybe being female and being so vocal.”

She talks about her creativity as something exhilarating and powerful, but also very fragile, liable to damage if she doesn’t handle it with care. “You work your whole life to build up integrity and you lose that in seconds if you allow yourself to do the done thing, or not listen to an instinct because someone in the industry has got an idea about who you might be for them. You’re defined by your choices so you have to be aware of that.”

Talking of choices, we meet a few hours after Theresa May announced the snap election. I would guess Tempest is near the top of the list of artists that one might approach to help galvanise young voters, but such an invitation would be a waste of time. She had an activist phase a decade ago, but became disillusioned and has not changed her mind.

“I’m not interested,” she says. “We’re in a terrible situation in this country and I don’t think any of us are quite prepared for what the next few years might bring. I think that the artist’s role is to be observant at all times and do their best to create and feed back. In terms of party politics, I feel like we’ve gone too far now for I’m right and you’re wrong. And I think if you align yourself too closely with a system that you find completely flawed then you’ll probably end up living to regret it.”

Last year, at the Sydney Writers’ Festival, she spoke powerfully about Australia’s “damaging and poisonous racism”, but then implored her audience to stop clapping. “My fear with public speaking is that at the moment it’s applauded that’s the work of the speech done,” she explains now. “I wanted to suspend the moment of discomfort for as long as I possibly could.” She shrugs off its significance. “A poet speaks at a literary event. This is going to change nothing.”

We talk some more about politics and she leaves the room for a moment. She returns with an anthology of Paris Review interviews with playwrights and reads from the introduction by critic John Lahr. He’s talking about the communal, participatory nature of theatre versus the solitary technological escapism that “tickles a society to death”. “The result is a palpable psychic mutation: a passive, credulous, restless mass, at once overexcited and underinformed,” she reads. “At its best, theatre is an antidote to the whiff of barbarity in the millennial air.” She pauses meaningfully.

If Tempest were a politician, her campaign slogan would be: Empathy, empathy, empathy. The more you notice about other people’s lives, the more you understand. “I think it’s important to play closer attention, to look again, to recognise that you’re part of something much bigger than yourself,” she says, energised again. “I find that extremely comforting. If at any moment your own narratives become too loud, just blink twice and look around and I guarantee you’ll be blown away by what you see. There’s so much life.”

Yet the idea a journalist might, in good faith, want to understand her does not fly with Tempest. As soon as we’re done, she doesn’t so much leave the room as escape it, dashing out for a cigarette. Out there, amid the city’s pulsing throng, she can again become the observer rather than the observed.