“I think in a Southern city, Southern town, white supremacy is woven into the American DNA,” said Rev. Seth Wispelwey of the local United Church of Christ. “There’s a lot of unreconciled history that has gone unchallenged.”

Saturday’s gathering marked the fourth time since May that white nationalists have gathered to protest Charlottesville’s decision to remove the statue of Confederate General Robert E. Lee and rename parks dedicated to Confederate leaders.

The statue’s location in Emancipation Park — formerly known as Lee Park — has become a meeting place for the conservative movement mixing racism, white nationalism and populism known as “alt-right.” This right-wing activism comes amid a renewed push across the South to remove Civil War-era symbols and names from public places.

Wispelwey, who has lived most in Charlottesville most of his life, and other locals were still processing the shock that washed over so many after white protesters and counter-protesters clashed in the heart of the city.

Most upsetting was when a silver Dodge Challenger drove into a crowd of counter-demonstrators, killing a 32-year-old woman and injuring more than 30 people. Soon after, police arrested 20-year-old James Alex Fields Jr. of Ohio, and charged him with one count of second-degree murder, three counts of malicious wounding and one count of “hit and run attended failure to stop with injury.”

“We have to talk to each other, but we also have to work to address the real problems that exist in the foundations of our community.”

Wispelwey said he began his Saturday by leading a group of clergy and faith-based leaders on a march. He, along with Jill Williams, a local teacher who marched with another clergy group, said they avoided areas where white nationalists were congregating and instead gathered with locals to counter hate groups.

Sean Clinchy, who lives in downtown Charlotte, said he felt it was important for residents to stand up and be counted.

“A lot of us felt that you had to show up and show there were more of us than them,” he said.

A number of locals who spoke with NBC News felt a need to participate in peaceful counter-rallies, but did not expect the violence even though they have witnessed months of building tensions.

A young woman sits near a makeshift candlelight vigil for those who died and were injured when a car plowed into a crowd of anti-fascist counter-demonstrators marching in Charlottesville, Virginia, on Saturday.Win McNamee / Getty Images

Brian Calhoun was one of those who began handing out water and offering bathroom breaks to those walking on the street in front of his house. Living just a block away from Emancipation Park still does not make Calhoun nervous, but he admitted to feeling afraid for the first time after seeing the Dodge Challenger ram the crowd.

“We’re going to get through this,” he said about his community. “But it seems like most of the violent protesters are coming to Charlottesville.”

He added: “Why is this happening here?”

Residents seemed eager to answer that question by forcing blunt conversations and acknowledging the social and economic divisions that slice through what Calhoun described as a “tiny, artsy, peaceful town.”

Two residents pointed to decades of gentrification as a reason why many low-income African Americans have felt shunned by their own city.

People gather during a vigil in Charlottesville, Virginia, on Saturday night.TASOS KATOPODIS / EPA

Charlottesville’s city council voted to remove two statues, but there has since been a delay due to a pending legal case, which has allowed white nationalist to use the issue as a focus of protests.

Though many white nationalists, neo-Nazis and radical counter-protesters came from outside Charlottesville, residents admitted white nationalism was embedded in the community.

Blogger Jason Kessler, who organized Saturday’s “Unite the Right” rally, is a Charlottesville local. Richard Spencer, the man who is considered to have coined the term “alt-right,” went to school in Charlottesville’s University of Virginia.

Now that the beginning of the school year is approaching, Williams, the teacher, said she was hoping other local educators would help students have open and frank discussions about what happened.

“The guy who drove the car reportedly was 20 years old,” she said. “You know I teach high school and we need to talk about this with the kids — all of it.”

Still, Williams said she was afraid that some teachers would feel it was best to pretend the whole thing hadn’t happened.

Though a somber feeling lingers over Charlottesville, all those interviewed said they were not fearful about living in their town.

“‘No hate, no fear,’ that’s what we’ve been chanting,” said Williams, adding that wasn’t the whole answer.

“We have to talk to each other, but we also have to work to address the real problems that exist in the foundations of our community,” she said. “If all we do is say ‘let’s love,’ there’s more work to be done than that.”