"The fact is that IMF is trying to get permission from everybody to sell gold...if and when IMF sells its gold, gold prices may go to a bottom," Rogers told Bloomberg.

"It may go down to U.S. $700,” Rogers says. “IMF has got a lot of gold to sell. If it does, I hope I'm brave enough and smart enough to buy more."

Though Rogers owns some gold, he’s not buying more because the IMF, which is one of the largest owners of gold in the world, “is desperate to sell its gold.”

Economist Judy Shelton says the IMF has been trying to ensure its own survival by seeking permission to sell its gold shares.

“While IMF officials insinuate the receipts would be used to help poor countries, the real goal is to set up a permanent endowment fund for the IMF,” Shelton writes in The Wall Street Journal.

The U.S., Shelton says, should not replenish the coffers of “a multilateral bureaucracy that quite literally lost its reason for being” when President Richard Nixon ended the Bretton Woods agreement that allowed countries to convert their dollar holdings, via the IMF, into gold at a fixed price.

“Congress should call for the IMF's dismantlement and restitution of its assets,” she says.