The last time the Dow was up over 150 points at the session highs, but closed down for the day was March 16, 2009.

The S&P 500 lost 13.81 points, or 0.8 percent, to finish at 1655.35. The Nasdaq dropped 38.82 points, or 1.11 percent, to close at 3463.30. Both the Dow and S&P were up as much as 1 percent earlier in the session.

All 10 S&P sectors fell, with utilities and materials pacing the declines.

June Taper Possible?

A number of participants express willingness to reduce QE in June, with CNBC's Steve Liesman. Bill Gross, Pimco, weighs in.

Fed Minutes Hint at Tapering

The Fed minutes noted that "a number of participants expressed willingness" to reduce QE as early as the June meeting if the economic data received by that time showed evidence of sufficiently strong and sustained growth. But officials differed about what evidence would be necessary and the likelihood of that outcome, the minutes said.

In his Congressional testimony earlier in the day, Bernanke said the central bank's "intention to maintain highly accommodative monetary policy as long as needed" and that any decision on scaling back bond purchases were possible in the next few meetings depending on an improvement in the jobs data.

Pimco's co-CIO Bill Gross told CNBC, "I think we're looking at a potential tapering in the next few months and probably around September."

Steven Ricchiuto,Mizuho Securities; and David Kelly, JP Morgan Funds, provide their views on whether the Fed's chairman will begin to taper its bond-buying program.

Last week, San Francisco Fed president John Williams suggested the central bank could begin hitting the brakes on its easing as early as this summer, sentiments that were echoed by Philadelphia Fed president Charles Plosser and Chicago's regional chief, Charles Evans.

But dovish comments from St.Louis Fed president James Bullard and New York Fed president William Dudley seemed to dampen expectations the Fed could pull back on its bond-buying soon.

JPMorgan Funds chief global strategist David Kelly, told CNBC the Fed should wind down its program, saying QE is too extreme. "The best thing for them to do right now is to talk about an exit strategy to gradually get people used to the idea that rates will go up," Kelly said. "That way people can go adjust away from some of the extremes that we're seeing in markets right now."

The next Fed policy meeting runs June 18-19.

In economic data, U.S. existing home sales data for April rose 0.6 percent to an annual rate of 4.97 million units, the highest level since November 2009.

Strength in Big Pharma

Pfizer said it will spin off its majority stake in animal health business Zoetis to shareholders.