From early July through the end of last week, bank shares had gained about 1.3 percent and outpaced the broader market by half a percentage point as investors anticipated that when the Fed raises rates, a move that could happen as soon as September, banks will make more money on loans.

But a rate hike in September seems a little less likely now. The yuan weakened for a second straight day on Wednesday after China devalued its main reference rate on Tuesday, and the currency hit a four-year low against the dollar, further reducing inflationary pressure in the United States and potentially spurring the Fed to wait until December or later to raise rates.

A decline in U.S. overnight indexed swap rates indicated the market sees a less than 50 percent probability of a rate hike next month, down from a near 60 percent probability after last week's solid U.S. jobs data.

Veteran bank analyst Dick Bove said the selloff in bank stocks over the past couple of days is clearly driven by changing expectations about what the Fed will do. "The market today and yesterday has made the assumption that there will not be any Fed hike this year," he said.

Many investors are still bullish on bank stocks for the medium or longer term. The shares are by some measures cheap relative to the broader market, with the S&P 500 trading at about 17 times estimated 2016 earnings, compared with big banks like Bank of America Corp, (BAC.N) Citigroup Inc, (C.N) and JPMorgan Chase & Co, (JPM.N) which trade closer to around 10 or 11 times expected earnings.

"Other parts of the market are at peak earnings and peak valuations so why wouldn't you want to rotate into depressed earnings on reasonable valuations?" said BlackRock portfolio manager Peter Stournaras. Portfolios he oversees for the $4.72 trillion asset manager have been overweight financial stocks for a few years, though to varying degrees. He last upped his overweight position following a selloff in January. Banks and managed care stocks are the largest exposures in his portfolio, he said.

Stournaras expects to remain overweight bank stocks for a few more years. "I don’t consider it a short-term trade. If they outperform by 50 percent it would cause me to review the position," he said. Another issue that might give him pause is if he felt "the Fed wasn’t going to ever budge," in lifting short-term interest rates.