There are millions of conservative African Americans in the United States, but Republicans have not "done a very good job" of taking the party's message into the nation's communities, Oklahoma Rep. Tom Cole said Tuesday.

"It takes understanding," the Deputy Majority Whip told MSNBC's "Morning Joe" program. "There's roughly 20 million African Americans in this country. I can assure you we should not be losing that vote 95-5 [percent]."

"There are millions of African Americans that are conservatives, that are evangelical in their faith, that served the country in the military, they are small business people," Cole continued. "We have a lot to say to those people. We might not win the communities but you don't have to lose them 95-5. It's a big difference getting beat 65-35 and 95-5."

Meanwhile, GOP candidate Donald Trump has recently started making a push to attract black and Latino voters, and Cole conceded Tuesday there are not nearly enough African American Republican lawmakers.

"I'm proud we have two in the House and one in the United States Senate," said Cole. "That's more than we had when I arrived in 2002. We've got quite a few, frankly, great Hispanic representatives and quite a few members who represent districts with very substantial Hispanic population. Those members have been awfully good about reaching out in their respective communities and establishing their own community."

Cole said he has spent a great deal of effort on Native American issues, as there are "11 Indian tribes in my district and about 8 or 9 percent of the state of Oklahoma are tribal members."

He does believe there is an effort by the Republican National Committee to reach out to diversify voters, and incumbents are doing more work on the issue than newcomers.

"It's hard to show up in a minority community two or three or four months before an election and say, 'hey, vote for me. Things will be different,'" Cole said. "Again, that's more than the Republican Party needs to do. I wouldn't dispute that for a minute because I think there are millions of African American, Hispanic conservatives that we haven't made a connection with. That's our fault, not theirs."

Cole supports Trump, but he said he would not take the "what in the hell do you have to lose" approach Trump has when it comes to appealing to black voters.

"I would say, things can be a lot better than they are," Cole said. "These are communities that we know aren't doing as well as broader communities, that do have higher unemployment, that do have less in the way of educational opportunities. ... I think when you do that you can find that you've got a lot more friends in those communities than you think but you've got to get up and make the effort."

Meanwhile, Cole said this election season will be tough for Republicans, but the odds are, "we'll hold the House," even though Trump's poll numbers are dropping.

"Both these candidates are pretty unpopular," said Cole. "Hillary Clinton will have a hard time in a way ... other candidates, [Libertarian Gary] Johnson and [Green Party nominee Jill] Stein, break up the vote at the top and people may well vote for those candidates in protest and revert to their normal patterns down ballot."

Cole also said he is not in the business of advising presidential candidates, and he won't do that "out loud and on television" with Trump when it comes to his immigration plans and the speech he plans for Wednesday.

"I think the important thing is that he tomorrow lays out a pretty clear vision of what it is that he's proposing and stick with it," said Cole. "The big thing is to make sure borders are secure and the problem doesn't continue to fester. They don't have a lot of faith in Hillary Clinton on that. They do think Trump is serious on that point."

There are millions of conservative African Americans in the United States, but Republicans have not "done a very good job" of taking the party's message into the nation's communities, Oklahoma Rep. Tom Cole said Tuesday.