When the prime minister, David Cameron, said on Wednesday that the riots had brought out "some of the best of Britain", he could not have known how fitting that description would be of events on a petrol station forecourt in a Birmingham suburb 12 hours later.

Spilling out in the road beside the Jet garage, where three Asian men were killed on Tuesday night, about 300 Muslim and Sikh men gathered to debate how they should respond to the tragedy.

There were no politicians in sight, no community spokespeople or religious leaders. These were local men, struggling to manage their grief and anger.

Candles marked the spot where Haroon Jahan, 21, and brothers Shazad Ali, 30, and Abdul Musavir, 31, had died. The three were part of a group of about 80 guarding the petrol station, on Dudley Road in Winson Green, when they were victims of a hit-and-run. A murder inquiry has been launched.

After prayers and a silent vigil, men took turns to express their views. There had been fears that the meeting held after the last Muslim prayer of the day, Isha'a, would be a flashpoint, sparking more rioting.

It was clear from snatches of conversation that there were a few in the crowd who wanted revenge on the black community, whom they held responsible for the deaths. They did not prevail.

It is hard to explain how the men reached their conclusion. After half an hour, the consensus among most was that there should be no march, in part because it would be disrespectful to the families of those who died. Not everyone agreed, and it was impossible to know whether dissenters would break away later and seek retaliation.

However community relations in Birmingham play out in the weeks to come, the meeting on Dudley Road will serve as evidence of the determination among many not to allow violence to spiral.

Standing on the wall of the forecourt, one Sikh man, Harpreet Singh, 28, began by imploring others not to take to the streets. He said they had gathered to pay their respects to the deceased and to prove they were united. He announced that the families of the dead did not want the group to march on the city centre, as had been planned.

"We need to tell the media we will not tolerate the tyranny, but we will not react either. We are capable, but we will not do it," he said.

He concluded that there were two possible outcomes – that they would protest, and the media would label them "extremists", or that they would act "nobly" and be seen as a community united. He added: "You decide. I will stand with you all the way."

Some in the crowd, both Muslims and Sikhs, agreed. Others did not, insisting they had come to protest.

"I say peacefully march, man," one voice shouted. Another said: "Let's do a march – but keep it peaceful."

An hour before the debate started, the atmosphere had been highly charged, as groups gathered for silent prayers for the dead men.

The crowd consisted mostly of young men, many of whom had hoods pulled over their heads – a handful with scarves concealing most of their faces. There were older men, too, and some community elders.

Police kept a low profile at the edge of the gathering. A few held their hats under their arms as prayers were said.

Speaking before the debate about whether to protest, Shaheen Kayani, 46, a cab driver from nearby Hodge Hill, said the Muslim community was pulling together to prevent trouble.

"Everybody says to their sons: please don't start another riot. People don't want trouble any more. I just say we want peace, peace, peace." Some of the younger contingent sniggered as he spoke. They were the same teenagers shaking their heads later, as the tide of opinion turned against street protest.

Some urged a reaction. "They've killed them for free, bro," said one. Others made racist comments about the looters. But when a man pointed his finger in the air, shouting: "We are going to protest to let them know how we feel," he was shouted down.

After more debate, one man stepped forward and lifted his voice above the murmurs: "Make sure you're not marching in the name of the three brothers that died. Because if you're gonna march… in their name, and you're rioting, it is a disgrace."

There were grunts of approval. Several people said they would go home. A handful of the masked youths walked away. Those who remained stood in near silence, heads bowed as they listened to Sikh and Muslim prayers.