But the New York Fed, which is JPMorgan's main regulator, cannot conduct a thorough investigation with Dimon on its board, said Jonson. "The public can't believe that an honest investigation can occur or that real accountability can ever come from it," he said.

Once one of the most admired CEOs on Wall Street, Dimon has come under fire from all corners after JPMorgan disclosed a massive $2 billion loss on trades that Dimon attributed to "sloppiness."

The loss has made JPMorgan a lightening rod for outrage over risky bets at big banks at a time when the economy is still recovering from the 2008 financial meltdown.

Among the other high-profile figures calling for Dimon to relinquish his Fed post are Elizabeth Warren, a former top economic advisor to President Obama. U.S. Treasury Secretary Tim Geithner has also suggested that having bankers like Dimon on Fed boards creates a negative perception, which is worth fixing.