With the passing of time, it's easy to become complacent about police racism. Nostalgic even. Ah, institutional racism: the eras of TV satires Life on Mars and Ashes to Ashes, when police called women slags and black people wogs. Happy days. Well no, of course not. It's worth reminding ourselves what institutional racism meant and, in some cases, still means today.

At its simplest, institutional racism – as defined by Lord Macpherson in his 1999 inquiry into the murder of Stephen Lawrence – means the treatment of people differently simply because of their colour or ethnicity. The report's findings implied that many officers may have felt they had more in common with Clifford Norris (the gangster father of Stephen's co-murderer David Norris, who was alleged to have paid corrupt police contacts to obstruct the investigation) than with Stephen and his friend Duwayne Brooks. Institutional racism meant that, despite the fact police were given names of five suspects within 24 hours, no arrests were made for two weeks.

Worst of all, institutional racism meant that the officers on the scene could not, or would not, accept that two black boys had been attacked and one of them killed simply because they were black. There had to be a motive – they had to be bad boys or members of a gang. And this is when the smears began, and where we saw institutional racism at its most vile.

The problem for the Metropolitan police back in 1993 was that try as they might they just could not get anything on Stephen and Duwayne. Neither had criminal records, they were at school and college respectively, and had not been involved in gangs – they were horribly respectable, in fact.

Stephen was always going to be harder to smear than Duwayne because, tragically, he was dead. Duwayne was an 18-year-old survivor, suffering from post-traumatic stress disorder. At home, he would barricade himself in his flat, stacking hi-fi equipment and heavy furniture behind the door for fear of "reprisals", too distressed to sleep. On the streets, he looked everybody up and down, making mental notes just in case they attacked and he had to give evidence against them.

If he was paranoid, he had every right to be. From the night Stephen was murdered, Duwayne was treated more like a suspect than a victim of the attack. When he told the police the killers had shouted "What? … What, nigger?" they made it clear they did not believe him. On the same night at the station, he was labelled "arrogant" and "surly", accused of taking a can of Coca-Cola from the police fridge without permission, and suspected of having broken a window. That same year, he was one of only a few charged with violent disorder after a demonstration against the BNP in Welling. This too was institutional racism, but it was to get worse.

Ironically, it was just after Macpherson's report labelled the Metropolitan police as such that the Met proved the point. Duwayne was arrested time and again on trumped-up charges. On two occasions he was accused of possession of offensive weapons which turned out to be part of his work kit as a photocopier engineer. In August 1999 he was accused of taking and driving away a car – his own.

I first met him a month later, just after his house was broken into and it emerged that the covert camera placed in his home for his protection had no tape in it. Duwayne told me at the time he had a feeling he was going to be arrested again on another trumped-up charge – something bigger, more damaging this time. Ten days later, he was charged with attempted rape. He was bailed for £20,000, had to leave London, and was placed under an 11pm curfew. Back then, that's what sometimes happened to young black men who were an irritant to the police – they got charged with a sexual offence. Almost a year later the case came to court – the charge by then had been reduced to indecent assault – and was thrown out before the defence even made its case. It emerged that the complainant had been pressured by the police to press charges, even though she had said she didn't want to and, when questioned, admitted Duwayne hadn't assaulted her; he had "just sat there".

The fact Duwayne was charged, and that the police were so determined the case would proceed that they told the complainant she could face arrest if she didn't go ahead or changed her story, all went back to the same starting point. The Met was desperate to prove, seven years after the murder, that Stephen and Duwayne were "rude boys", "ragamuffins" or worse. There could, after Macpherson's report, have been another public inquiry into Duwayne's treatment that would surely have reached an equally damning conclusion.

Institutional racism is by no means the exclusive preserve of the police. When the public began to believe the smears against Duwayne, some former supporters turned against him, for fear he might not, after all, be the untainted victim needed to head up their campaign for justice for Stephen. For years he was not invited to events, private and public, commemorating the death of his best friend.

And yet somehow, Duwayne emerged from it all with tremendous dignity. After Stephen's death he administered a charity that ran hostels, and in 2009 became a Lib Dem councillor in Lewisham working in the community on safety schemes that prevent gang violence. He has always been stroppy and opinionated, but also strong and loyal with a pure moral core. These qualities emerged at the Lawrence trial, when Duwayne stood in the witness box – hours after his father had died – to describe the attack. As he recalled the blood streaming from Stephen's neck and jacket and his dying friend repeatedly asking "What happened to me? Look at me", he broke down in tears, taking almost a minute to regain his composure. Prosecutor Mark Ellison QC offered to summarise that part of the evidence, but Duwayne slapped his hand on the rail of the witness box, raised his voice and said: "No, I want to say what happened." His graphic description of the killing, which he had previously described in his memoir Steve and Me as being "like a lynching from the days of slavery", helped secure the conviction of Norris and Gary Dobson.

In recent days, there has been a good deal of back-patting by the media, the judiciary, the police and campaigners – with some justification: the fight for convictions was relentless. But at the same time, there has been a worrying sense of revisiting a bygone era. In legal circles, it has actually become known as the "Life on Mars" defence: yes, we know these appalling things happened, but not any more.

Has British society really changed so much, though? The week before the convictions of Norris and Dobson, an Indian student, Anuj Bidve, was killed in Salford in what is being investigated as a hate crime. Football is often cited as an area of progress, notably with the Kick It Out campaign, yet Liverpool's Luis Suarez is serving an eight-match ban for racism, and England captain John Terry is due to appear in court for the alleged racist abuse of Anton Ferdinand.

As for the police, many notable examples of the police's apparently racist assumptions occurred after the death of Stephen Lawrence and, disturbingly, post-Macpherson.

When Roger Sylvester died in 1999 after being restrained by at least six police officers he was portrayed as a feral, naked black man prowling the streets of Tottenham; in fact, he was an average-sized naked man with mental health problems locked outside his house. He was also described as a crack addict, although no traces of cocaine were found in his blood or urine.

Mikey Powell, the cousin of the poet Benjamin Zephaniah and a man without a criminal record, died in 2003 after police officers drove their car at him before spraying him with CS gas and restraining him. It was claimed that police had driven their car at him because he pointed a gun at them. He was actually holding a belt. When the family complained to West Midlands police, they were told it had been a mistake made by a source close to the investigation. It so happened that the picture they painted of him again fitted the racist stereotype.

After Jean Charles de Menezes was shot by police on the tube, we were informed that he was a "suspected terrorist" wearing a suspiciously heavy or padded jacket, with wires sticking out, who had run away from the police and jumped a ticket barrier. The subsequent police apology for this misinformation did little to pacify the family.

Institutional racism dead? Try telling that to the family of Mark Duggan, who heard that police only shot him dead (an incident that was the catalyst for last year's riots) after he first fired at them, hitting the police radio – a version of events that turned out to be nonsense.

In Macpherson's report, he defined institutional racism in terms of how Duwayne had been treated by the police after Stephen Lawrence's murder. "We believe that Mr Brooks's colour and such stereotyping played their part in the collective failure of those involved to treat him properly and according to their needs."

"Dear Mr Brooks,On behalf of the Commissioner I would like to apologise for the manner in which you were dealt with by police, following the murder of your friend, Stephen Lawrence, and the attack on you. You were let down. The Macpherson Inquiry identified police shortcomings and I wish to reassure you that important lessons about the proper treatment of victims and witnesses have been learnt.It is also a personal regret of the Commissioner and I that those who attacked Stephen and you have not yet been successfully prosecuted."

In the past week, the Metropolitan police and the British judicial system has gone some way towards righting part of the wrong. And yes, some things have improved in Britain and, specifically, in the Met: by 2009, one in five recruits was from black and ethnic-minority communities (though, worryingly, there are no officers at the level of assistant chief constable or above, whereas there were four 13 years ago at the end of the public inquiry); the double jeopardy law was abolished (a Macpherson recommendation) allowing Dobson to be tried a second time for the same crime; and the police were told they had to investigate an incident as racist if victims or witnesses perceived it to be so.

Yet there is still huge distrust of the police – and some things have actually got worse. In 1999, black people were four-to-five times more likely to be stopped and searched by the police than white people. Today, black people are eight-to-10 times more likely to be. It is still the case that not a single police officer has been successfully prosecuted for negligence or corruption in the Stephen Lawrence investigation, nor for excessive use of force leading to the many deaths in custody of black men and women over the past 40 years.

And data from the pressure group Inquest shows that, since 1991, eight out of the 11 people who were ruled to have been unlawfully killed by police were black (three of the 11 verdicts were subsequently quashed or overturned, and none of the 11 resulted in successful prosecutions of police). Until the British justice system makes it clear that police officers will go to jail for discriminatory behaviour, it is surely premature to talk about police racism in the past tense.

Simon Hattenstone is the ghostwriter of the book, Steve and Me: My Friendship with Stephen Lawrence and the Search for Justice, by Duwayne Brooks

• This article will be open to comments from 11am (UK time) on Saturday