Amazon should start an online bank. I'll try to explain why I think they're in a good position to do so.
Of course, some people still need a physical high-street bank. But there's a huge market for online-only banks. Few companies have the expertise, position and philosophy to offer such a good online banking service as Amazon.

Why Amazon?

A huge customer base. Clearly it makes marketing easy, but I think the bigger benefit comes from knowing its customers. Fraud is a major problem for any company that handles money, but given its list of long-term customers with known spending patterns, Amazon credit cards and verified addresses, Amazon can eliminate a whole set of possible fraudulent activities. Amazon has experience handling fraud, and the computational power and technical ability to build systems to detect it even better.

Excellent customer service. It's something most banks (in my experience) are terrible at, and that Amazon is famous for. To the point that some people (including me) will buy things off Amazon simply to avoid any potential hassle. If something goes wrong, Amazon will deal with it - they give their support staff the power to fix problems.

Safety. Customers want their money to be secure, and their finances kept private. Amazon has a wealth of experience in online security, keeping their cloud services (e.g. AWS) safe.

Experience with meeting regulations. Amazon has already been certified in many different areas for specific government needs, and provides services that have been certified for processing credit cards and storing health data. Banking is heavily regulated, but I don't think it will be a problem for Amazon.

Cash. Because of the aforementioned regulation, setting up a bank is (probably) quite hard, and will take a lot of lawyer-time and money. Startups may have difficulty entering this market, but that's not a problem for Amazon. They are, after all, quite big.

Experience online. An "online" - "bank". Traditional banks don't have the experience of running large, easy-to-use websites, Amazon does (though obviously it lacks the experience in banking). Although many of the services Amazon offers are extremely technical and complex, Amazon's customer interfaces are usually extremely easy to use. They (temporarily) patented the one-click checkout method. Not that making a website better than some banks is hard.

Innovation. Recently we've seen a glimmer on innovation from a few banks, such as Barclays Pingit, and Canada's Mintchip . But most seem to think that email and text alerts is the high-point of banking and technology. Amazon has enormous room for improving the way people use and look after their money. Next-day delivery of foreign currency? Bought at a price you ask for? Sounds good :)

Integration. Amazon could integrate its banking service with its other products. It could avoid credit card processing fees and do bank transactions, even passing the savings on. Imagine a bank account that gave you 3% off anything you bought at Amazon! I say 3% since their existing credit cards offer 2% already.