The president has faced criticism for his response to white supremacists who gathered in Charlottesville, Virginia last weekend to protest the removal of a statue of Gen. Robert E. Lee. The rally turned into a violent clash with some of the counter-protesters and a white supremacist plowed his car into a crowd, killing a female protester and injuring 19 others.

Huckabee said on "America's Newsroom" that some monuments help to understand history, even "hideous" points in American history.

"Where does it stop?" he asked, adding "anarchists" cannot be allowed to show up and knock down statues, as they did on Monday in Durham, N.C.

The former Arkansas governor said that tearing down historical statues that might offend people is not the answer.

"There's something to be said for looking at our history - the good, the bad and the ugly - and being reminded of that which we never want to repeat again," said Huckabee, relating a story from a recent visit to Russia.

He said state and local governments should make decisions on monuments in their own communities.