Judges walk from Westminster Abbey to the Houses of Parliament in 2012 to mark the start of the legal year. Photograph: Oli Scarff/Getty Images

The ceremony is one of the defining images of the British constitution: judges in their wigs, scarlet gowns and gilt regalia processing from a service in Westminster Abbey across the road into parliament for the lord chancellor's breakfast.

The official opening of the judicial year takes place again on 1 October. A tradition dating back to the middle ages, it provides a visual reminder of the formal link between church and state.

The question of whether an independent judiciary should – in an ever more multidenominational and secular society – continue to bend its knee in public before an Anglican altar has now been raised with the justice secretary, Chris Grayling.

In a letter calling for a separation of the Church of England from judicial affairs, a former long-serving Ministry of Justice official and a one-time Tory parliamentary candidate have now proposed pursuing legal action if the event is not ended.

Peter Fisher, recently retired from the MoJ, and John Butcher, now a Surrey councillor, fear the service "prejudices judicial decisions on religious matters" and have asked for the names of the judges attending the abbey since 2007 to be published.

In medieval times, they point out, it was customary for judges to seek "divine guidance" for judicial decisions. Judges are at risk of seeming biased in cases involving a witness or defendant in which an issue of religion arises, they argue. "The judge trying such a case is placed in a difficult position if [he or she] has attended the judges' service [and] may appear to have prejudged the religious issue by publicly appearing to support particular beliefs," their letter states.

Judges attending religious services in a personal capacity do not cause any problem, Fisher and Butcher maintain.

They add: "In Scotland there has long been a related tradition of holding two judges' services at the start of each legal year: a Protestant service in the kirk, and a Catholic 'red mass'. That practice is now a topic of public controversy in Scotland, where it is currently under debate, and so may well soon have to be changed."

The best solution, they advise Grayling, would be to cancel the service in the abbey and continue with the lord chancellor's breakfast in Westminster, since that has no religious element.

Next Tuesday's procession will be led for the first time by the new lord chief justice, Sir John Thomas.

Fisher told the Guardian that he knows "for a fact" that some judges are quite uneasy about [attending the abbey service]. Judges are reluctant to complain about it, however, in case it adversely affects their careers.

Butcher said: "To attend the service in public, in judicial time, in this way really does compromise their independence." One way of resolving it, they warn, would be a judicial review of the practice – decided by a judge who had never previously attended the service.

One religious question that will be the subject of a supreme court ruling in the coming weeks is whether Scientology buildings qualify as places of worship entitled to conduct weddings.

Attendance at the Westminster Abbey serrvice is not compulsory for judges. A Ministry of Justice spokesperson said: "We believe concerns about possible religious bias resulting from the judges' service are completely unfounded.

"On appointment every judge is required to take both the oath of allegiance to the Crown and the judicial oath which requires them 'to do right to all manner of people after the laws and usages of their realm without fear or favour, affection or ill will'.

"Judges attach the greatest importance to this oath and reach decisions without partiality or bias of any kind, including on matters of religious faith."