(CNN)Sen. Lindsey Graham says Donald Trump is a "wrecking ball" who has put the Republican Party's future on the line with his controversial remarks about Mexican immigrants.

"I think he's hijacked the debate. I think he's a wrecking ball for the future of the Republican Party with the Hispanic community and we need to push back," Graham said in an interview with CNN's Dana Bash on "State of the Union" Sunday.

"This is a defining moment for the Republican Party. We need to reject this," he said.

JUST WATCHED

Graham grew up in a bar, lost parents at young age

MUST WATCH

The South Carolina Republican who is competing against Trump and more than a dozen others for the party's presidential nomination is among the GOP's biggest proponents of immigration reform.

Read More

He said he'd still like to double U.S. border patrol on the U.S.-Mexico border and triple the number of drones monitoring the area.

But he said Trump -- who he called a "demagogue" -- is hurting the party's ability to court an increasingly important voter bloc in key swing states.

"I'm very worried about where we're headed as a party. I don't think this is the way to get the Latino vote," Graham said. "If we do not reject this way of thinking clearly, without any ambiguity, we will have lost our way. If we don't reject it, we've lost the moral authority, in my view, to govern this country."

Trump has stuck by his assertion that most Mexican immigrants are criminals, and has insisted he'd build an impenetrable wall -- which he'd make the Mexican government pay for.

Those comments came as he shot near the top of Republican primary polls. Graham, meanwhile, is still trying to gain traction to crack the top 10 in the polls. Only those 10 will make the main stage for the first GOP presidential debates.