Barely 3% of Britain’s most powerful and influential people are from black and minority ethnic groups, according to a broad new analysis that highlights startling inequality despite decades of legislation to address discrimination.

The numbers betray a grotesque disconnect with the composition of the UK population, almost 13% of which has a minority background. In some sectors – the police, military, supreme court and security services as well as top consultancies and law firms – there were no non-white supremos at all.

Equality advocates said the new study shone a light on the glass ceilings, subtle discrimination and “affinity bias” that minorities face as a matter of course in their careers. The toll is severe, on individuals, communities, and society as a whole, they said.

“We need to ensure that every young person has a role model they can look up to,” said the London mayor, Sadiq Khan, one of the 36. “It’s so important to promote the successful figures from Britain’s BAME communities. We need to create a sense of optimism, aspiration and hope.”

Speaking at a Guardian Live event at the Labour party conference in Brighton on Sunday, Khan also identified confidence as a key factor in holding BAME people back. “Some white people feel confident about applying for jobs even if they are not fully qualified. But BAME people may not have the same confidence even though they are more qualified,” he said.

The data analysis looked at the ethnicity of more than 1,000 individuals across 39 categories covering politics and the civil service; policing, defence and the judiciary; FTSE companies and groups representing business; professional services including the heads of law, accountancy, advertising, consulting and publishing firms; arts bodies; media; trade unions; top universities; sporting bodies and NHS trusts.

The results of the exercise call into question the effectiveness of equality legislation such as the 2000 Race Relations Act and the 2010 Equality Act, which have seemingly done little to promote people of minority backgrounds into the highest positions of power.

How did we create the list of Britain's 1,000 most powerful people?

Read more

“Pathways to power are almost non-existent if you’re black or Asian,” said Simon Woolley, director and co-founder of Operation Black Vote, which helped compile the Colour of Power data along with the recruitment consultant Green Park.

“The white club virtually locks out black talent. The lack of diversity at the top level is deeply troubling, not least because in most sectors there has been little or no progress at all.”

Rebecca Hilsenrath, chief executive of the Equality and Human Rights Commission, said the research showed that “inequality and unfairness are still entrenched in our society”.

“Without real diversity in leadership positions we will never be truly reflective of society, particularly when it comes to public services and bodies which serve our communities. There is absolutely no reason that BAME people should not be able to reach the top of their chosen profession,” she said, calling on the government to put in place a comprehensive and coherent race strategy.

The shadow equalities minister, Dawn Butler, said black people were losing out because contacts and networks trumped talent. “If people do not fulfil their potential in all walks of society then we all lose out: the country loses out, that person loses out, that business loses out.

“As a black woman nothing surprises me when it comes to representation of both people of colour and women … It’s really not because of lack of talent. That’s the disappointing thing.”

Play Video

2:25

The colour of power: why is the British establishment so white? – video

The latest data amplifies a report published earlier this year by the businesswoman Ruby McGregor-Smith, which found that while 14% of the working age population were from a BAME background, they made up only 10% of the workforce and held only 6% of top management positions.

There has been some progress further down the ranks since the 2000 Race Relations Act which, among other things, placed a duty on public bodies to promote racial equality.

There have also been some advances in politics: 8% of MPs after the 2017 general election are of minority ethnic backgrounds, the highest ever proportion, while the current cabinet has two BAME members compared to the all-white 2001 cabinet under Tony Blair.

The Inequality Project: the Guardian's in-depth look at our unequal world

Read more

But despite this progress, various reports have concluded that inequality remains deep-rooted in society. A recent TUC survey found that more than a third of BAME workers reported bullying, abuse or experience of racial discrimination in the workplace while the recently published Lammy review warned of racial bias in the criminal justice system in England and Wales.

Dr David Owen of the Warwick Institute for Employment Research in Warwick University, whose research has looked at the changing ethnic makeup of the population and the labour force, said evidence showed workplace barriers were manifold.

“The evidence from analysis of data over many years is that minority ethnic groups are less well represented in higher status occupations than white people,” he said.

He said the issue of diversity at the highest levels mattered for a number of reasons, most importantly that people of BAME backgrounds should feel they are able to fulfil their potential without “glass ceilings or other barriers to their ambition”. He added that it was also an advantage to organisations to best “understand the population and the society they are serving”.

While the project concentrated on BAME diversity, it also showed up a startling gap in gender representation. Women account for less than a quarter of the names on the list (23.6%). Among FTSE 100 chief executives, there are more men called David than women.

Have your opportunities for career advancement been restricted because of your ethnicity? Please share your thoughts with us using the form below: