"The tax system is a disgrace," said Rubenstein, speaking at the Council of Institutional Investors spring conference in Washington, D.C., on Monday. "When you have inequity in the system as we do now, it's not good for the country as a whole."

Earlier this year, Romney's tax return revealed that the former Massachusetts governor paid an effective tax rate of just 14.5%.

According to the documents, Romney took in $21.7 million in long-term capital gains over the past two years, and of that, $12.9 million was in so-called carried interest. Income from carried interest is taxed at the preferential rate of 15%, instead of the top income tax rate of 35%.

President Obama's tax plan calls for Americans who make at least $1 million to pay their "fair share" in taxes, which would include paying ordinary income tax rates on carried interest.

"I think it's a little unfair for people to say you're not paying your fair share of taxes," said Rubenstein. "I'm paying what I'm supposed to pay. Change the law, and I'll pay what I'm supposed to pay."

He added that the debate over carried interest will likely be resolved in the next Congress as lawmakers seek to overhaul the U.S. tax code.

"The inequities should be dealt with," Rubenstein said. "[Carried interest] has gotten a lot of attention and it's a sore point for people. It's no doubt that Congress will address it in comprehensive tax reform."