However, Lockhart's moves didn't t actually violate any rules. Fed presidents are only restricted from trading for the seven days that precede a Federal Reserve policymaking meeting.

Losing Confidence in the Federal Reserve

Still, such appearances of impropriety "undermine confidence in the system," said Money Morning Capital Waves Strategist Shah Gilani, Editor of the Wall Street Insights & Indictments newsletter. "People think, "What's the point? The system is rigged against me.'"

Given the Fed's mandate to control inflation, the investments of Dallas Fed president Richard Fisher also raised a few eyebrows.

Fisher, who spent 22 years in the financial industry as a banker, stockbroker and hedge fund manager, owns at least $1 million in gold in the form of the SPDR Gold Trust exchange-traded fund (NYSEARCA: GLD). Fisher is also invested in platinum and uranium.

While not suggesting new restrictions on the investing habits of Federal Reserve presidents, Gilani does think full transparency would help restore the public's flagging confidence in the Fed.

"When they buy and sell should be publicly disclosed immediately, within a few days of the transaction, not months later," Gilani said. "Why not do it?"

More Conflicts at the Fed

The Federal Reserve had already attracted attention for a different sort of conflict of interest.

Last fall the Government Accountability Office (GAO) warned of concerns from having high-ranking executives from private banks serve on the boards of the Federal Reserve's regional banks.

Those private banks, of course, are regulated by the Fed and can get emergency funding from it.

The structure dates back to the 1913 origins of the Federal Reserve, but has come under more scrutiny since the massive bailouts of the financial sector in 2008.

"It's a very bizarre structure that arose as a political compromise a long time ago," Kevin Hassett, director of economic policy studies at the American Enterprise Institute and a former Fed economist told Bloomberg News. "There's this sort of shadowy world where our financial regulators are connected with Wall Street. It's really important that the New York Fed not be viewed as a captive of Wall Street."

The GAO investigation also showed that New York Fed president William Dudley owned shares of American International Group (NYSE: AIG) and General Electric Company (NYSE: GE) during the financial crisis. Both of those companies got help from the Fed.

Dudley, who reported his holdings to the Fed and eventually divested them, said last fall that Federal Reserve officials have recognized the problem. How much of it will get fixed – and when — is another question.

"We have to take appearance of conflict really seriously because it does affect the institution by creating questions about our credibility," Dudley told Bloomberg News.

Staying out of the markets is the worst financial decision you can make. Investing can have a phenomenal impact on your life, and this resource has everything you need to get started. Enter your email below to get the report.

Money Morning gives you access to a team of ten market experts with more than 250 years of combined investing experience – for free. Our experts – who have appeared on FOXBusiness, CNBC, NPR, and BloombergTV – deliver daily investing tips and stock picks, provide analysis with actions to take, and answer your biggest market questions. Our goal is to help our millions of e-newsletter subscribers and Moneymorning.com visitors become smarter, more confident investors.