Hundreds of supporters of the Cairo football club, which lost 74 fans in Wednesday's riot, have rallied outside the club's premises to mourn the dead and call for revenge against fans of their rival team.

As the rally took place, fans of Al Ahly club in central Cairo were still burying those killed in the northern city of Port Said during the clashes with supporters of their bitter rivals, Al Masri. The violence was Egypt's worst football riot in 15 years.

Yet, among the outrage of the crowd of about 400 people, a sense of responsibility was stirring. "Take that sign down," said one leader of Al Ahly ultras, the militant fans at the centre of Wednesday's disturbances. "We don't want to blame Scaf (Supreme Council of the Armed Forces) today. Politics can wait."

Another group argued about whether to chant political slogans on a march to Tahrir Square planned for late afternoon. "I'll chant what I want, when I want it," shouted one man, but those arguing against political statements won.

The ultras, who have been on the frontline of clashes with riot police in Cairo for the past year have had an uneasy relationship with Egypt's liberals. As well as battling riot police at football fixtures, the ultras have led a series of reinvigorated protests in recent months against Egypt's military rulers, which has renewed focus on the waning liberal push for influence in the evolving post-Mubarak society.

The country's silent majority, many of whom backed the status quo and looked disdainfully on the liberals as naive utopians, has – until now – been even more scathing of the ultras, who they see as reckless anarchists.

But the manner of the 74 deaths – and where they died – has resonated with many Egyptians, much more than clashes in Tahrir Square or the killing of 27 Coptic Christians outside the government broadcasting headquarters last year. Football strikes an emotional chord that crosses sectarian and social structures in Egypt.

"They shouldn't have been killed like that, no matter who they are," said Ali Abbas, who supports the military leadership and believes it will deliver on a promise to see Egypt through the transition to civilian leadership. "The violence does not appear normal. It seems like punishment."

Mohammed Salama, 23, an Al Ahly ultra whose leg was broken in the stadium riot, said it became clear at half-time in the match between the two historical foes that trouble was brewing.

Leaning on a red walking cane he said Al Masri supporters had stormed through open gates after full time and trapped him and other fans against locked gates at the back of the stadium.

"They threw me off," he said, pointing at his leg in a full plastercast. "They were saying: 'You should have brought (burial) shrouds to the game.'" Another Al Ahly fan said the same words were displayed on a sign outside the grounds before kickoff. "It became clear that they were planning an ambush," he said. "It had to have been backed. The gates [to the pitch] are never open like that."

Tahrir Square was again heaving with demonstrators, many of whom buy into the Al Ahly view that militant Al Masri fans were given a green light by some elements of the security forces to attack their rivals.

The streets near the interior ministry building were again a battleground between riot police, several hundred of whom held a frontline near their headquarters on Friday, and mostly young Egyptians who ran the gauntlet, throwing rocks, molotov cocktails and, according to the security forces, sometimes shooting weapons. Four people died on Friday clashes, two in Suez, near Port Said, and two in Cairo. Medical authorities in the capital said around 1,500 people had been treated for injuries in the past 48 hours. Most appeared to have suffered from teargas inhalation. The Al Ahly ultras say they will rejoin the fray when the time is right. "When we do, everybody will know we are there," said one ultra Mahmoud Saleh. "These fans did not die for nothing."