More from Fox
Local police estimated that at least 6,000 people are in attendance. Among those expected to attend are Confederate heritage groups, KKK members, militia groups and "alt-right" activists, who generally espouse a mix of racism, white nationalism and populism.
Kessler said this week that the rally is partly about the removal of Confederate symbols but also about free speech and "advocating for white people."
"This is about an anti-white climate within the Western world and the need for white people to have advocacy like other groups do," he said in an interview.
Charlottesville Mayor Michael Signer said he was disappointed that the white nationalists had come to his town and blamed President Donald Trump for inflaming racial prejudices with his campaign last year.
"I'm not going to make any bones about it. I place the blame for a lot of what you're seeing in American today right at the doorstep of the White House and the people around the president."
Senate Minority leader Chuck Schumer tweeted before Trump's reaction: "March & rally in Charlottesville against everything the flag stands for. President Trump must condemn in strongest terms immediately
First Lady Melania Trump also tweeted: "Our country encourages freedom of speech, but let's communicate w/o hate in our hearts. No good comes from violence. #Charlottesville."