Friends and family of Lewis Elwin, who was fatally stabbed in April 2016, stage an anti-knife crime march before his funeral in Tooting, south-west London.
Photograph: Lauren Hurley/PA

On the afternoon of 18 April 2016, a Monday, Lewis Elwin was dropped off by a relative in Moyser Road, a residential street in Tooting, south London.

The 20-year-old’s mother and five older siblings were worried about him: they feared he was mixing with the wrong crowd, and doing so at a time when tit-for-tat violence among youth gangs was costing lives.

But this was considered a safe area: a tree-lined street of Edwardian family homes, coffee shops and a flower shop. Elwin called a friend and they arranged to meet.

He walked 200 yards to the south: by the time he turned left into Penwortham Road he had broken into a jog. By this time, police now believe, Elwin had already sustained the knife wound that had pierced his chest. He collapsed, and died within minutes.

On the first anniversary of Elwin’s death, his friends gathered at his graveside, where they danced and sang. Some wore T-shirts bearing his picture beneath the message “RIP Lewis”.

Also present was a young film-maker who filmed and recorded the group. Using a camera on a drone, he also shot footage high above the 14-storey tower blocks of a housing estate in Tooting where some of the young men lived.

Some can’t even cross the road without putting themselves into a dangerous location

Ebony Reid, sociologist

The film was edited and handed over to Elwin’s friends, who posted it on YouTube on 21 April 2017. Now, thousands of people could see that a number of Elwin’s friends were members of a gang that called itself 417. They could not see their faces, however: all but one were wearing balaclavas. But they could listen as the gang members sang about how they planned to “ching ching” and “wet their opps” – stab rival gang members. “March to the opp block, samurai down my leg,” they sang.

The following night, members of the gang drove to Wandsworth, four miles’ north of Tooting. They were armed with a number of weapons, including a 27-inch (69cm) sword. In the early hours of the morning, they cornered Mahamed Hassan, 17, who was cycling past and stabbed him several times.

Mahamed’s cries alerted a resident who called an ambulance. Asked to check for injuries, the man lifted the boy’s top and saw that his intestines were hanging out. The boy died a short time later in hospital.

A knife found at the address of one of the four teenagers jailed for the murder of Mahamed Hassan Photograph: Metropolitan police

Four of the attackers were in custody within days and went on trial for murder at the Old Bailey at the start of the year. When all four were found guilty, one, Tyriq Aboagye, was clearly surprised. “How can you find us all guilty,” he shouted at the jury. “Let me know coz, tell me bruv, the fuck you think it’s like bruv.” As he was led away, still screaming, the judge, Nicholas Cooke QC, turned to the jurors and said: “I think you’ll understand I see this quite often.”

The four teenagers were each jailed for life. Their tariffs – the minimum terms they must serve – totalled 85 years between them. Aboagye was given 27-and-a-half years.

The taking of two young lives and the waste of four others is another example of the apparently senseless violence gripping London in particular over the last two years: there have been more than 89 murders in the capital so far this year, most of them knife crimes. Serious youth violence increased by 9.6% in the 12 months to April in London; knife crime by 18.3%.

Tyriq Aboagye. Photograph: Metropolitan Police

But a small number of criminologists, sociologists and youth workers have been attempting to make sense of “senseless” youth gang violence. They find that while gang members’ lifestyles may initially appear bewildering, it was possible to see a logic and a structure in the choices these young people make and the violence they inflict and suffer. Separately, they were also coming to strikingly similar conclusions about the complex causes of that violence.

Unsurprisingly, many of them point to the economic and social factors that result in a small number of young people becoming marginalised, and then attempting to make their way in life in the brutal and unpredictable world of the drug trade.

More surprising, perhaps, was the conclusion that many gang members were traumatised and frightened young men – and that the most vulnerable were frequently the most dangerous.

Last year Ebony Reid, a sociologist at London Metropolitan University, published a PhD thesis based on her study of 29 gang members on the north London housing estate where she grew up. She called the estate Northville, and the gang’s members called themselves the mandem, a widely used slang term for a group of men or boys.

Reid found that 17 of the 29 had been permanently excluded from secondary school and two others had attended pupil referral units. Only three completed A-levels or further education. Four had part-time jobs. The remaining 25, including all but one of those excluded from school, were unemployed.

Reid concluded that the gang’s members were trapped in the drug trade because they could not find work. “A precarious drug economy in Northville offered various menial jobs and some lucrative career opportunities for its participants,” she said.

I was sent to a school for bad kids. I had a lot of free time. I got sucked into the group lifestyle, the street lifestyle

Ex-gang member

Last month a report published by Waltham Forest council in north-east London noted schools’ concerns that the word “gang” should not appear in any of their Ofsted inspection reports, and pointed to the high proportion of gang members who had been excluded from school.

A former member of a London gang described to the Guardian how he was excluded and sent to a pupil referral unit at 12.

“I just had one fight, outside of school. We was from the same school. He was severely injured. He said that I stabbed him, but I didn’t. I punched him and his tooth went through his face.

“I was sent to a school for bad kids. I had a lot of free time. I was there in the morning from nine until 12, and then I was out. We didn’t wear uniform, my mum was at work, my dad was at work, I was left to my own devices. I would meet up with all the kids who were in the same situation as me, been kicked out.

“I got sucked into the group lifestyle, the street lifestyle. I was young, I wanted to be popular, I wanted to be respected by the older boys. I did what I thought was right to get the credentials I wanted. Fighting, showing my strength, and being feared in the area. This is when I was 12.”

At 14 he was stabbed seven times. A year later he was jailed for 18 months for his role in an affray in which another teenager died.

By the time Aboagye was adopted at seven he had had a number of homes and was said to already be deeply troubled. He too was sent to a pupil referral unit. “That was a disaster,” his adoptive mother Jennifer said. “He’s a bright boy, but he’s gullible, and he wants to be liked, he wants to be loved.” By the time he was arrested for murder, Aboagye already had 15 convictions. “I don’t like to think about the length of his sentence,” she said.

He’s a bright boy, but he’s gullible, and he wants to be liked, he wants to be loved

Mother of Tyriq Aboagye

In her thesis, Reid concluded that the mandem were trapped not only in drug dealing but also in their neighbourhood, because they feared being attacked if they strayed into a rival gang’s territory. “Some can’t even cross the road without putting themselves into a dangerous location,” she said.

This postcode rivalry is rooted in real fear. The film-maker who made the 417 video – and who has made many other similar films of gang members – said he had met many who have never been outside their neighbourhood.

“Some guys in Tooting have never left Tooting, for example. When I have suggested shooting videos in, say, Westminster, they would say: ‘Bro’ I’ve never left Tooting, there is no way I’m going to Westminster’.

“One of them actually said to me: ‘I don’t want to go to Westminster and see opps’. They think if they go to Westminster they would have to carry a knife. They don’t know that there’s no gangs in Westminster. It’s crazy.”

Finally, Reid judged, the mandem were trapped in a “hypermasculine” sense of identity. “There is a need to be perceived ‘on road’ as tough or bad, and yearning to retaliate when masculinity comes under attack. To survive requires a certain kind of mastery: defend your manhood or become a victim.”

The young men she studied would defend their manhood by “earning stripes”, which in practice meant attacking a member of a rival gang.

Floral tributes to Kelvin Odunuyi, outside a cinema in Wood Green, north London. Photograph: Martin Godwin for the Guardian

Gavin Hales, a criminologist who has worked closely with the Metropolitan police, agreed that boys and young men in the most deprived communities could be acutely anxious about their status.

“The young men concerned often have a particularly fragile sense of self-worth” and being shown disrespect was often given as a motive for assaults and even murders, he wrote. “In this world, reputation is absolutely critical and signs of vulnerability may be ruthlessly exploited by others … to allow someone to disrespect you without consequences is to admit weakness and invite trouble.”

The anxiety gnawing at these young men has been particularly acute after a killing. In Wood Green, north London, last March, after the fatal shooting of Kelvin Odunuyi, a 19-year-old who rapped as DipDat (stab that), teenagers stiffened in fear whenever they saw an unfamiliar figure approaching. “Anything could happen at any time,” said one. “All people have to do is go into the kitchen and get a knife, it’s so easy.”

Four weeks later and two miles to the east in Tottenham, on the Northumberland Park estate, the territory of Odunuyi’s rivals, the streets were deserted for several days after the fatal shooting of Tanesha Melbourne-Blake, 17. The motive for that shooting remains unclear. “You can really feel the tension, can’t you?” said one young man hurrying past.

The motive for the fatal shooting of Tanesha Melbourne-Blake, 17, is unclear. Photograph: LONDON METROPOLITAN POLICE / HANDOUT/EPA

A series of surveys have established that the main reason young people carry knives was for self-defence. “For a young person whose life is dominated by fear and survival, carrying and even using a knife may seem completely rational and indeed essential,” said Hales.

Perhaps the most difficult factor to unpick, when attempting to make sense of “senseless” youth violence, was the current influence of UK drill, the rap music that is a cousin of Chicago drill, and whose lyrics not only describe gang warfare, but sometimes include threats and taunts against particular gangs and/or individuals.

A number of rappers have been murdered in London in recent years, and postcode feuds have been fuelled by “diss tracks” posted on YouTube and other sites where they quickly gather thousands of views. The music that 417 recorded at Elwin’s graveside was drill music, for example, and had been viewed almost 50,000 times by the time Aboagye and his friends were jailed for life.

There has been no end of headlines about “the brutal rap that fuels gang murder”. But does the music arise out of youth violence, or fuel it, or both?

After another Old Bailey trial, at the end of which two gang members were jailed for life for stabbing a music producer to death, Cooke, again presiding, observed: “Music, of course, does not kill people. Music is a great cause for good, whatever genre or type.” He added, however, that there was legitimate concern about the glamorisation of violence in some of the videos that feature in the murder cases he had been hearing.

Ciaran Thapar, a writer and youth worker in Loughborough Junction, south London, described drill as “a war report on life in the urban trenches of austerity Britain”. While acknowledging that it could normalise violence, and could also contribute to acts of violence, Thapar said the young men making the music endured deep-rooted problems involving family life, communities and state education.

“Most, if not all the young men making drill are angry, neglected and have no investment in formal society. That is the reason why there is a connection. The music reinforces its source, so let’s cut the source.”

Reid said that even more important than giving gang members economic opportunities was the need to broaden their horizons, and to offer them counselling for the anxiety they suffered.

This is not what has been happening. Instead, in May, YouTube took down about 30 music videos after a request from the Met. The police said they ask that videos be taken down only when they raised the risk of violence, and also used a range of criminal behaviour orders such as curfews and non-association orders to limit gang violence.

Is UK drill music really behind London's wave of violent crime?

Read more

Last month, drill musicians who were members of a gang, and who had admitted conspiring to cause violent disorder, received a court order that banned them from making music without police permission.

Nobody has been more critical of these measures than the members of that small group of people who have been attempting to make sense of gang violence, and who fear that attempts to suppress drill will lead to the further alienation of those young men who regard the music as the soundtrack to their difficult lives.

Hales was particularly critical of the tendency of politicians and senior police officers to latch on to easy solutions. “Youth violence is a long-term, entrenched problem,” he said. “It can’t be solved by a quick fix.”