Dozens of people demanding better pay and conditions for Walmart workers were arrested at a protest in Los Angeles which organisers called the single biggest act of disobedience against the retail chain.

Police detained 54 people on Thursday night, including Walmart workers, union representatives and clergy members who sat in the street outside a Walmart store in Chinatown and refused to move, prompting officers to declare an unlawful assembly and move in.

“Walmart has proven its willingness to break the law by illegally firing workers and trying to silence them,” said Maria Elena Durazo, executive secretary-treasurer of the Los Angeles County Federation of Labor, one of the organisers. “We are sitting down today to demonstrate that we won't allow these dirty tactics in Los Angeles.”

The demonstration was the latest in a year-long series of mostly small, vocal protests at Walmart stores around the US.

A woman is arrested during a protest for better wages outside Walmart in Los Angeles. Photograph: Lucy Nicholson/Reuters

OurWalmart, a union-backed members group, says as many as 825,000 Walmart workers – out of a workforce of 1.3 million – are paid less than $25,000 a year.

It accused the US's biggest employer, which recorded profits of $17bn over the past year, of keeping workers in poverty and punishing and firing those who spoke out. The group says it has filed more than 100 unfair labour practice charges with the National Labor Relations Board, including 20 illegal terminations and 80 disciplinary actions.

Organisers said a fifth of the 500 people who attended Thursday's protest were Walmart workers.

Los Angeles police move in to arrest people during a protest for better wages outside Walmart in Los Angeles. Photograph: Lu

Richard Reynoso, an overnight stock worker at the chain's store in Duarte, was one of those detained. “I got arrested today because I believe that taking this step will encourage others to be brave and step forward and stand up to the world’s largest retailer,” he said. “Walmart can’t silence me.”

Kory Lundbeg, a Walmart spokesman, said the company estimated there were only 15 to 20 employees in the crowd and that none worked at the Chinatown store. External agitators were driving the protests, he said. “The reason is our associates understand the benefits of working for Walmart.”

The chain promoted on average 430 employees daily and recruited three-quarters of its management teams from the shopfloor. A store manager earns $170,000. Stores in Los Angeles county operated as normal on Thursday, and the company expected to work without disruption through the holiday season, Lundberg added.

Denise Barlage, 54, a clothing room operator at Walmart's Pico Rivera store in LA, disputed the company's depiction of contented workers and said many were overworked, stressed and intimidated.

“A lot of the people I know are fearful. It's like walking on rice paper. They feel at any moment they may be fired.” Fear of retaliation deterred complaints about pay and conditions, she said. “Everyone is just so nervous all the time a lot don't speak up for themselves.”