"I think the fact that she's seen an increase in risk-taking is probably causing the market to give up some gains," said Peter Cardillo, chief market economist at Rockwell Global Capital.

Investors may not fully appreciate the risk of future losses, Yellen warned on Wednesday, citing falling corporate bond spreads and volatility indicators as possible indicators. She said there was no need for a change in monetary policy focus in the meantime as it is not the best financial stability tool.

"That said, I do see pockets of increased risk-taking across the financial system, and an acceleration or broadening of these concerns could necessitate a more robust macroprudential approach," Yellen said in prepared remarks for a speech at the International Monetary Fund.

May factory orders reported a greater decline than expected on Wednesday morning.

"I don't think it's inconsistent with an improving economy," said Art Hogan, chief market strategist at Wunderlich Securities. "The decrease is basically in line with expectations."

"I would certainly expect of the data today that ADP is more impactful than factory orders," he said.

Stocks were mixed despite a better-than-expected ADP jobs report, which is viewed as an indicator of the all-important non-farm payrolls due Thursday, estimated to rise by 212,000.

"I suspect all depends on tomorrow's employment data," Cardillo said. "The premium that the market is seeing due to geopolitical concerns right now is pretty much at its peak. We've seen a short-term high in oil prices. I think investors should be slightly cautious at these levels."

The dollar (Exchange:.DXY) edged higher against other global currencies and the U.S. 10-year Treasury (U.S.:US10Y) note yield increased slightly to 2.62 percent.

The Dow Jones Industrial Average was within 35 points of 17,000 early Wednesday afternoon after missing the high by less than 2 points on Tuesday.

"17,000 is a great round number," said Hogn. "We should be there already."

Poor performance on new additions to the Dow have contributed to the index's sluggish rise compared to other indices since it crossed 16,000 last November.

Shares of camera maker GoPro (NASDAQ:GPRO - News) traded lower on Wednesday after days of consecutive gains that boosted prices to double the initial offering of $24 a share.

Google (NASDAQ:GOOGL - News) shares dipped slightly after announcing the purchase of streaming music service Songza late Tuesday.

Rackspace Hosting (NYSE:RAX - News) shares jumped nearly 8 percent to $36.44 after TechCrunch reported the cloud service provider may take itself private and was in talks with a private equity firm to fund the deal.

Shutterfly (NASDAQ:SFLY - News) shares jumped nearly 14 percent to $49.49 after Bloomberg reported that the company was working with boutique investment bank Qatalyst Partners to find buyers for itself.

JPMorgan Chase (NYSE:JPM - News) shares were down about 1 percent after Chief Executive Officer Jamie Dimon said he had been diagnosed with early stage throat cancer but would remain actively involved in the largest U.S. bank's business while in treatment.

Adding to a recent string of transactions that has lifted stocks in the healthcare sector, Roche (Swiss Exchange: ROG-CH) said it would pay up to $1.725 billion to buy Seragon Pharmaceuticals, a privately-held U.S. biotech company that researches breast cancer treatments.