A bubble currently brewing in sovereign debt will likely burst in the next couple of years, U.S. billionaire Wilbur Ross warned on Monday.

"I've felt for some time that the ultimate bubble, when we look back a few years from now, is going to be sovereign debt, both U.S. and other, because it's way below any sort of reversion to the mean of interest rates," the distressed debt investor told CNBC.

"If you look at where the U.S. 10-year had averaged over the 10 preceding years, it's around 4 percent. If it reverts back to that level at some point there will be terrible losses in the long-term Treasury market and those will probably be accentuated in other areas of fixed income."

Ross argued that slowing issuance of assets like mortgage-backed securities and long-term Treasurys post-credit crisis, had helped to insulate the market from the full impact of the Federal Reserve's gradual slowdown of quantitative easing - a process known as tapering.

Investors have to "build in refinancing risk" when buying assets at the moment, he said.

The amount of cheap money in the market, as a result of quantitative easing by both the Fed and its European counterpart the European Central Bank (ECB), has been credited with the resurgence in investment in assets like peripheral euro zone bonds.

Ross, who recently sold his entire stake in Bank of Ireland, said he had very recently put money into Greece and had also invested in Virgin Money in the U.K.

He bought the Bank of Ireland shares at 10 cents in 2008 and sold then for 26 and 27.5 cents earlier this month. The investment had become "too big a part of our portfolio", he said, adding "it was already a pretty big position when we started". Ross said it was not because of "dissatisfaction" with the Irish economy.

"We need to be in situations where there is stress," he said.

The billionaire investor also attacked the U.S.'s tax policy, after a slew of high-profile companies exploited or tried to take advantage of overseas tax loopholes. One recent example is Pfizer's $106 billion bid for U.K. rival AstraZeneca.

"The way politicians should deal with it is to lower the U.S. rate...then we wouldn't need all this jurisdiction shopping for tax," he said, adding that it is an "anomaly" rates are so high.

"I think they're taking more risk by not doing anything," he said of U.S. politicians - but added that the chances of anything happening before elections in November were slim.

"That money is not going to come back. Do you really think Apple or Google is just going to bring their money back?"

Ross argued that the U.S. government was going "a bit far" in its crackdown on banks like BNP Paribas, and that the risk of hefty fines was putting banks off lending.

He added that valuations for companies which are being acquired are "getting pretty out there."