Judges today ruled that one of Britain's most successful faith schools had racially discriminated against a 12-year-old boy who was refused admission because the school did not recognise him as Jewish.

In a landmark legal decision, supreme court judges found that the Jewish Free School, a comprehensive in north-west London, had broken the law by refusing to admit the boy, a practising Jew known as M.

It had denied him a place because it has twice as many applicants as it can take and prioritises children whose mothers are recognised as Jewish by the Chief Rabbi.

M's mother converted from Catholicism to Judaism under a non-Orthodox authority, meaning the Chief Rabbi does not recognise her as Jewish. He only recognises children as Jewish if he recognises their mothers as Jewish.

The boy's father took the school to court, claiming racial discrimination. In June the court of appeal ruled in his favour, saying the school's policy amounted to racial discrimination because it prioritised applications from children with Jewish mothers.

The school appealed, taking the case to the supreme court. Critics said today's ruling would mean secular judges are deciding who is Jewish and who is not.

The ruling will lead to children who apply to one of the UK's 50 Jewish schools having to sit religious practice tests to ensure the schools are not discriminating against them on ethnic grounds.

It is also likely to lead to a revision of admissions policies at faith schools.

Lawyers said it was the most controversial ruling since the supreme court was created in October.

The judges ruled, by a majority of five to four, that the school had "directly discriminated against M on grounds of his ethnic origins" and was in breach of the Race Relations Act.

Reading out the judgment, Lord Phillips, the president of the supreme court, said: "The supreme court has dismissed the appeal by the governing body of JFS.

"The majority held that JFS had directly discriminated against M on grounds of his ethnic origins."

Phillips said the judges did not consider the Chief Rabbi to be racist. The judgment "should not be read as criticising the admissions policy of JFS on moral grounds, or suggesting it was 'racist' in the pejorative sense", he added.

M's father, who cannot be named for legal reasons, said: "I believe it's important for people to know that the same Race Relations Act that provides such valued protection for Jews, as well as others, from ill-judged or misguided prejudices also provides for the fair and equal treatment of all children within our education system.

"It is very important to see that this essential protection was not mistakenly discarded by divisive views which can naturally occur from time to time within all communities.

"The Jewish community, which has long endeavoured to enshrine fairness and care for others, will be relieved at heart that this minor discord will be put aside and that we, like all God's children and people of true feeling, can pull together again and work to make a better and fairer world for all."

The United Synagogue, which represents Orthodox Jews in the UK, said it was "extremely disappointed" with the ruling.

It said the decision "interfered" with the "Torah-based imperative on us to educate Jewish children, regardless of their background".

"Essentially, we must now apply a non-Jewish definition of who is Jewish," Simon Hochhauser, the synagogue's president, said.

Trevor Phillips, the chair of the Equality and Human Rights commission, said the verdict confirmed that "no school will be allowed to discriminate based on the ethnic origin of an individual".

The British Humanist Association said the court ruling should trigger an investigation into all state faith schools' admissions policies.

"There's absolutely no reason why what is essentially a public service should be denied to any children, whatever their beliefs or the beliefs of their parents," Andrew Copson, the association's director of education and public affairs, said.

Rabbi Jonathan Romain, the chair of the Accord Coalition, which campaigns for inclusive education, said he hoped the ruling would serve as a "wake-up call", prompting faith schools to stop discriminatory policies.

The chairman of governors at JFS, Russell Kett, said the supreme court judges had suggested "there may be a defect in the law by not allowing the school to give admission priority, when oversubscribed, to those who are Jewish in Orthodox law".