The world's largest banks need to shrink or be broken up in order to regain investors' confidence after four years of scandals, high-profile trading losses and financial crises, according to a Bloomberg poll.

Almost 60 percent of respondents said they were not confident or "just somewhat confident" that banks are taking prudent risks and conforming to the law, and getting smaller was seen as the top fix in the Bloomberg Global Poll, with 29 percent choosing that remedy. Changing the compensation structure was the No. 2 way to improve trust, with 23 percent.