U.S. Republican presidential candidate Donald Trump on Monday rejected criticism – including from his own party – of his allegations of bias against a Hispanic judge, insisting his concerns were valid.

“All I want to do is figure out why I’m being treated unfairly by a judge. And a lot of people agree with it,” Trump said on Fox News.

Trump has been on the defensive since his comments last week about Mexican-American U.S. District Judge Gonzalo Curiel, who is overseeing fraud lawsuits against Trump University, the New York businessman’s defunct real estate school.

He has suggested that Curiel’s heritage is influencing his opinion about the case because of Trump’s campaign rhetoric about illegal immigration. Trump, the presumptive Republican nominee for the Nov. 8 presidential election, has pledged to seal the U.S.-Mexico border with a wall, and has said Mexico is sending rapists and drug dealers to the United States.

Trump has regularly stirred up controversy on the campaign trail and has frequently dismayed Republican establishment leaders. His view of an ethnically biased judiciary has drawn a fresh wave of criticism, including concern in his own party.

He doubled down on Sunday when asked if – by the same token – he believed a Muslim judge would be biased against him based on Trump’s call for a temporary ban on Muslims entering the country. “It’s possible. Yes,” Trump said on CBS’ “Face the Nation.”

Republican leaders including House of Representatives Speaker Paul Ryan and Senate Republican leader Mitch McConnell have distanced themselves from Trump’s comments, saying they are worried the tone of his presidential campaign could enrage Latinos, who are a growing U.S. voting bloc.

On Monday, Senator Jeff Flake expressed alarm.

“If this doesn’t change we’re in for big trouble,” the Arizona Republican said on MSNBC. “I hope to be able to support the nominee. I certainly can’t now,” said Flake, whose state has a large percentage of Hispanics.

A former rival for the Republican presidential nomination, Ohio Governor John Kasich, called on Trump to apologize to Curiel, who was born in Indiana to Mexican immigrant parents.

“Attacking judges based on their race (and/or) religion is another tactic that divides our country,” Kasich wrote on Twitter. “More importantly, it is flat out wrong.”

CLINTON SEIZES THE MOMENT

Hillary Clinton, who is hoping to clinch the Democratic presidential nomination in nominating contests on Tuesday, loses no opportunity to point out when Republicans disagree with their standard bearer. She jumped in on the controversy over Curiel.

“@realDonaldTrump’s bigoted comments about a Latino judge are so disgusting, even other Republicans are offended,” she said in a Twitter post.

Trump fought back against his Republican critics on Monday, calling it inappropriate for Newt Gingrich, a former House speaker, to weigh in on the issue.

Gingrich, who has been mentioned as a possible running mate to Trump, on Sunday called Trump’s comments inexcusable.

“I was surprised at Newt,” Trump told the “Fox & Friends” program. “I thought it was inappropriate what he said.”