Members of the KKK are escorted by police past a large group of protesters during a KKK rally in Charlottesville, Virginia, in July this year.Photograph: Steve Helber/AP

A white nationalist rally, which some have predicted could be the largest in the US in years, is expected to descend on Saturday on Charlottesville, Virginia, and be met by counter-protests.

Far-right monitoring groups estimate that between 500 and 1,000 people and 30 speakers and groups will descend on the downtown area for the afternoon event, organised by the local rightwing activist and former Daily Caller writer Jason Kessler.

Those attending the Unite the Right event will hear speeches from leading “alt-right” ideologues including Richard Spencer, the podcaster Mike Peinovich, AKA “Mike Enoch”, and Matthew Heimbach, leader of the Traditionalist Workers party.

There was a confrontation late Friday ahead of the main rally when protesters and a smaller group of counter protesters came together on the University of Virginia campus. When the marchers reached and surrounded the counter-protesters there was a short verbal confrontation. Counter-protesters said they were then attacked with swung torches, pepper spray and lighter fluid.

Speaking earlier, Spencer Sunshine, who wrote a report for Political Research Associates assessing Saturday’s rally, said: “This is a national gathering that the far right have been planning for months. It’s their big event.”

He said far from being a broad slice of the right as planned, with the reluctance of some “patriot movement” groups to attend, and the commitment of groups such as the National Socialist Movement to the event, “it’s become increasingly Nazified over the last few weeks”.

In response, local demonstrators and anti-racist activists from all over the country are coordinating a counter-protest, which they are hoping will dwarf the far-right event.

One group, Congregate Charlottesville, has called for 1,000 faith leaders, and “white clergy, especially”, to assemble for a nonviolent protest. Among those who have responded is the intellectual and activist Cornel West, who led a prayer vigil on Friday night, ahead of the rally.

Another group, SolidarityCville, issued a call in late July to “all anti-racists” to come to Charlottesville to oppose the event, adding that “we must outnumber them in order to #DefendCville”.

“Black Lives Matter and SURJ (Showing up for Racial Justice) chapters will also be opposing the rally.”

These calls have been amplified on a range of other progressive and antifascist websites.

Far-right activists have raised the prospect that the confrontation could turn violent. Discussing so-called “antifa” groups on his podcast The Right Stuff this week, Mike Peinovich said: “We don’t want those people to have the impression that we are going to show up unarmed.”

Emily Gorcenski, a local activist who is involved in organizing the #defendcville protests, said in an interview: “I don’t want violence, we don’t want violence, that’s not our MO. We want to be able to demonstrate against their message and assert our civil rights.”

But she also said that she had had 100 death threats in the last month, and was herself open-carrying a weapon for self-defense. “The second amendment is one of the few civil rights I have left as a trans woman.”

On Friday afternoon, police reportedly attended the carpark of the city’s Walmart, where they spoke with a Unite the Right speaker, Christopher Cantwell, after they received a report of someone brandishing a gun.

At the same time, police were supervising the construction of barricades around a 20ft-statue of the Confederate general Robert E Lee, which is the focus of the rally, and has been at the centre of other recent confrontations in Charlottesville.

In February, the city council narrowly voted to remove and sell the statue, and to rename the park in which it stands from Lee Park to Emancipation Park. This was the culmination of a campaign to remove the statue started by a local high school student, Zyahna Bryant.

This is part of a wave of such removals of Confederate monuments across the south, which began after Dylann Roof massacred nine African American churchgoers in Charleston, South Carolina.

In response, last May, Richard Spencer led a torchlit white nationalist parade around the park. Then, on 8 July, about 50 members of the Loyal White Knights of the Ku Klux Klan held a rally in the park, and were greeted by around 1,000 counter-protesters. The day ended in turmoil after police used tear gas on some counter-protesters following the Klan’s departure, and made 23 arrests.

Saturday’s rally is expected to go ahead in Emancipation Park despite the city’s attempts to relocate it to McIntire Park, which is outside the central downtown area. Organizers and the ACLU were challenging the city’s right to relocate the rally, and on Twitter Friday, Jason Kessler was insisting it would go ahead in what he was still calling “Lee Park”.

The Charlottesville mayor, Michael Signer, and the University of Virginia president, Teresa Sullivan, each issued statements urging people to avoid the rally.

Around the city starting on Thursday evening, flyers appeared bearing the slogan “Diversity is a code word for white genocide”, a common white nationalist trope.

The governor of Virginia, Terry McAuliffe, in a statement urged residents to stay away. He said: “I want to urge my fellow Virginians who may consider joining either in support or opposition to the planned rally to make alternative plans.

“Many of the individuals coming to Charlottesville tomorrow are doing so in order to express viewpoints many people, including me, find abhorrent. As long as that expression is peaceful, that is their right. But it is also the right of every American to deny those ideas more attention than they deserve. Men and women from state and local agencies will be in Charlottesville ... to keep the public safe, and their job will be made easier if Virginians, no matter how well-meaning, elect to stay away from the areas where this rally will take place.”