"But I’m also not going to validate the language Hillary Clinton used to describe the American people," Pence said on Tuesday. "She was talking about people across this country who are coming out in record numbers to stand by Donald Trump in record numbers."

Pence told NBC News people with "ill motives can associate" themselves with whoever they like in free country, and compared Duke’s support of the Trump ticket to the Orlando shooter's father appearing at Clinton rally last month.

Clinton, the Democratic nominee for president, initially claimed at a fundraiser that half of Trump’s supporters could fit into a “basket of deplorables.”

After coming under criticism, she issued a statement over the weekend that she regretted using the word “half.” What is “deplorable,” she said was that Trump hired Breitbart News’ Steve Bannon, “a major advocate for the so-called ‘alt-right’ movement to run his campaign and that David Duke and other white supremacists see him as a champion of their values.”

When Blitzer pushed Pence on if he’d call Duke, who is running for the Senate in Louisiana, a “deplorable,” Pence answered, “No I’m not in the name calling business…” He added, “What Hillary Clinton did Friday night was shocking. I mean, the millions of people who support Donald Trump around this country are not a basket of anything.”

After the interview, Duke lauded Pence’s decision to not denounce him.

“It’s good to see an individual like Pence and others start to reject this absolute controlled media,” he told BuzzFeed , later adding “The truth is the Republican Party is big tent. I served in the Republican caucus. I was in the Republican caucus in the legislature. I had a perfect Republican voting record. It’s ridiculous that they attack me because of my involvement in that nonviolent Klan four decades ago.”

On Tuesday, Duke tweeted several references to the brouhaha.

"It's ironic that the people claiming to celebrate diversity are the people pushing to destroy the unique diversity of mankind," Duke tweeted on Tuesday.

Clinton meanwhile, previously pounced on Pence’s remarks, tweeting, “If you won’t say the KKK is deplorable, you have no business running the country.”

Her running mate, Tim Kaine echoed that sentiment at a campaign event in Ann Arbor, Michigan on Tuesday.

"If you cannot call out bigotry. If you cannot call out racism, xenophobia... If you cannot call out and you stand back and be silent about it then you are enabling it to grow. You are enabling it to become more powerful. We want to be a nation of the positive virtues not the dark emotions and not the negative virtues," Kaine said.

In a private meeting with Pence, Sen. Mike Lee-R-Utah, told the vice presidential candidate that the GOP must identify "Duke’s racism as deplorable," a spokesman for the lawmaker told NBC News.

"Sen. Lee spoke with Gov. Pence about his interview with Wolf Blitzer. Lee emphasized that Republicans must identify David Duke’s racism as deplorable, acknowledging that he ordinarily uses terms like 'deplorable' to describe messages, ideas, actions, and organizations, but not people," said Lee spokesman Conn Carroll. "This, Lee explained, is consistent with his belief as a Christian that even people who do bad things are redeemable. Lee also encouraged the Trump campaign to be explicit in its denunciation of the alt right movement."