The co-chairman of the Council on Foreign Relations added, "I thought we had a reasonable chance at a grand bargain," in the lead-up to the deal to avoid the "fiscal cliff."

Of course, that didn't happen, and he said that House Republicans did not engage with the president in the compromise process.

"Today there's virtually no bipartisan activity unless it's in response to a crisis or very special circumstances," Rubin argued, adding the polarization in Congress has "hardened substantially" since the era when Bill Clinton was president.

Comparing now and then, he told CNBC, "[If] President Obama faced the same political situation that President Clinton had, it would have been very difficult — just as it was very difficult for President Clinton." But he added, "Obama in a different style, in his own way, would have been equally effective working with the opposition."

—By CNBC's Matthew J. Belvedere; Follow him on Twitter @Matt_SquawkCNBC. Reuters also contributed to this report.