By 1 p.m. ET, police had cleared the park where the rally was to be held. It wasn't immediately clear how many demonstrators remained in other parts of the city.

"It is now clear that public safety cannot be safeguarded without additional powers, and that the mostly out-of-state protesters have come to Virginia to endanger our citizens and property," McAuliffe said. "I am disgusted by the hatred, bigotry and violence these protesters have brought to our state over the past 24 hours."

It wasn't immediately clear what led to the fights, though tensions and rhetoric were running hot. At one point, a few dozen white men wearing helmets and holding makeshift shields chanted, "Blood and soil!" Later, another group chanted slogans like, "Nazi scum off our streets!"

People punched and kicked each other during various scuffles, which often were broken up from within crowds, without police intervention, CNN video shows.

Earlier, a group of clergy and other counterdemonstrators, including activist and Harvard professor Cornel West, held hands, prayed and sang, "This Little Light of Mine."

Police presence was heavy, with more than 1,000 officers expected to be deployed, city officials said. Police anticipated the rally would attract as many as 2,000 to 6,000 people, and the Southern Poverty Law Center said it could be the "largest hate-gathering of its kind in decades in the United States."

White nationalists wield torches

Charlottesville, once home to Thomas Jefferson, is known as a progressive city of about 47,000 people. During last year's presidential election, 80% of its voters chose Hillary Clinton.

But far-right activists and Ku Klux Klan members have come here in recent months, outraged by the city's intention to remove traces of its links to the Confederacy -- including plans to remove a statue of Confederate Gen. Robert E. Lee.

The effort developed amid a push by communities across the South to remove Confederate iconography from public property since the 2015 rampage killings of nine black churchgoers in Charleston, South Carolina, by a self-described white supremacist.

Ahead of Saturday's planned rally, tensions roiled Friday night as white nationalists -- some holding what appeared to be backyard tiki-style torches -- marched onto the University of Virginia's campus.

Chanting, "Blood and soil" and "You will not replace us," the group rallied around a statue of Thomas Jefferson before they clashed with counterprotesters, CNN affiliate WWBT reported. The group left the university's grounds when police arrived and declared the gathering an unlawful assembly.

"In my 47 years of association with @UVA, this was the most nauseating thing I've ever seen. We need an exorcism on the Lawn," Larry Sabato, director of the university's Center for Politics, tweeted.

Charlottesville Mayor Mike Signer released a statement referring to Friday's rally as a "cowardly parade of hatred, bigotry, racism, and intolerance march down the lawns of the architect of our Bill of Rights."

"Everyone has a right under the First Amendment to express their opinion peaceably, so here's mine: not only as the Mayor of Charlottesville, but as a UVA faculty member and alumnus, I am beyond disgusted by this unsanctioned and despicable display of visual intimidation on a college campus," he added.