The first women arrived at the Western Wall as the dawn mist cleared, deftly binding their arms with black leather straps that fixed Torah scrolls to their fingers and heads. By 7am, there were 300 women clad in white prayer shawls and skull caps drawing stares of amazement as their singing swelled in a loud chorus.

A group of teenage schoolgirls praying nearby burst into fits of giggles. One wailed tunelessly, mimicking the women's enthusiastic songs. Another shook her head at the group of women praying like men. "You're crazy!" she shouted across a line policewomen, half amused, half outraged. "This is against the Torah!"

Sunday's service celebrating the first day of the Jewish month of Tamuz marked an historic moment in modern Jewish history. Since Israel reclaimed the Western Wall in 1967, one of the most sacred sites in Judaism has been run in strict accordance to ultra-Orthodox protocol. Women and men have been segregated. Only men have been allowed to sing from the Torah, don white prayer shawls and apply the black leather t'filin straps.

Sunday marked a victory for Women of the Wall, a feminist group which has been campaigning for the right to pray on equal terms to men at the site since 1988. The activists argue that Jewish law does not prohibit women from praying as men do. International support for their cause rocketed earlier this year, when 10 members – including the sister of the US comedian Sarah Silverman – were arrested for illegally wearing prayer shawls. In April, Israel's supreme court finally conceded their case and ruled that the Women of the Wall should be allowed to pray freely at the site. The decision has outraged Israel's largely conservative religious community.

Years of tension erupted in violent clashes and multiple arrests during prayers at the wall last month. On Friday, a teenage settler was arrested for inquiring on Facebook if it was permissible under Jewish law to shoot Women of the Wall members. But on Sunday, a police force that only weeks ago had arrested Women of the Wall supporters flanked them at every side. Dozens of armed officers ensured they were bussed directly to the entrance of a segregated corridor leading into a pen in front of – but not touching – the Western Wall. Ella Rembrand, a 33-year-old car mechanic, was among several women who refused to be "caged".

She crossed the barricades into the open women's section wearing a kippeh and attempted to embrace the ultra-orthodox women praying loudly beside her. The women recoiled, denouncing Rembrand and her fellow worshipers as "an abomination to God". From the far side of the wall, men poked their heads caped with tefillin boxes and broad-brimmed hats over the fence into the women's section. A young mother spat across her pram into the Women of the Wall enclosure.

"Take your trousers and go pray elsewhere," another yelled.

Undeterred, Rembrand argued that this hostile reaction proved her point. "We are trying to be inclusive, to encourage pluralism, and they are determined to stay in their own ghettos, both mental and physical," she said.

"The Kotel [the Western Wall] has been an ultra-Orthodox synagogue for years but now things are changing. I wear a kippeh because I am Jewish — more Jewish than [those ultra-Orthodox] are."

Sunday's ultra-Orthodox counter-protest was smaller than anticipated, with around 200 men, women and children gathered to protest against the controversial female worshipers, holding banners that read: "Provocative women! You have made up a new religion. Go ahead, build a new modern wall!"

Sara Rigler was among them.

"Women's singing is offensive here. Men who don't want to hear women singing can't close their ears. It's sexually provocative and violates local mores," she said. "This issue has been cast in terms of freedom. But it's not about the freedom of women to worship – this is violating the freedom of men not to hear women singing."

Acknowledging the angry chants echoing across from the men's section, Women of the Wall supporter Lior Nevo cradled her two-month daughter Netta and smiled: "It's the price of our success – at least they notice us now."

Praying within a barricade of police was strange, she said, but Sunday's service was undoubtedly a leap in the right direction.

"I was arrested here when I was eight-months pregnant. Last month, I came with my baby and the violence really scared me. Now the rabbinate has realised there is no sense in arresting us. In 12 years, Netta may have her batmitzva here,'' she smiled. "Israeli attitudes are changing."