Memo to Charlie Rangel: You’re in some pretty uncharted waters if former Gov. David Paterson — son of one of your closest friends and allies — is publicly taking you to task for your racially charged remarks.

The hoopla began with a Rangel slam against the Tea Party: “It is the same group we faced in the South with those white crackers and the dogs and the police,” the Harlem congressman told Politico. “They didn’t care about how they looked.” In other words, the Tea Partiers are the heirs of Bull Connor and the segregationists.

A few days later, as a guest on Mike Huckabee’s radio show, Rangel was treated to a thumping — not by an outraged Republican, but by Paterson. “I thought it was totally out of line for the congressman to lump what went on during the desegregation movement in with the people who just have honest political disagreements,” Paterson said.

Paterson went on to add that he didn’t find the Tea Party’s views on government spending and lower taxes “to be particularly strident.”

There’s an irony here that Paterson — the state’s first black and legally blind governor — may appreciate, even if Rangel does not. Far from being akin to the violent segregationists of the 1960s, the Tea Party has made today’s Republican Partymore diverse.

The Tea Party movement arose in 2009, played a key role in the 2010 and 2012 elections, and saw many of its candidates triumph. And who are these people?

They include Hispanic senators Marco Rubio in Florida and Ted Cruz in Texas; Indian-American Gov. Nikki Haley of South Carolina; Hispanic governors Susana Martinez of New Mexico and Brian Sandoval of Nevada; African-American Sen. Tim Scott of South Carolina (elected to the House in 2010 before being appointed by Haley to fill retiring Sen. Jim DeMint’s unexpired term).

In fact, thanks in part to the Tea Partiers, Republicans now make up two-thirds of the Senate’s Latinos, and Scott is the only African-American in the upper chamber.