Later, Fed Gov. Daniel Tarullo will be on CNBC at 10 a.m. EDT, and Dallas Fed President Rob Kaplan speaks at 9:30 a.m. at the Mission Capital conference in Austin. Kaplan also speaks in the evening at the Dallas Security Traders Association convention.

Markets have all but ruled out a rate hike for September after Friday's disappointing August jobs report and weak ISM data on manufacturing and services. But Fed speakers are now being watched to see if any push the idea that the Fed could hike soon, an idea raised both by Fed Chair Janet Yellen and Fed Vice Chairman Stanley Fischer before the weak batch of data.

"Every Fed speaker, you have to care about at some point, but I'm still hard pressed to see them raise rates in September, barring some great data," said Justin Lederer, rates strategist at Cantor Fitzgerald. Treasury yields rose sharply Thursday but held within a recent range. The 10-year was yielding 1.60 percent in late trading.

Lederer pinned the move on disappointment with the European Central Bank on Thursday morning. "I think people were expecting to see more on [quantitative easing] and didn't get it, and the rate complexes across the world are under pressure," he said. Bond yields move inversely to prices. He said besides the Fed speakers, the bond market will be watching the action in European yields and also Japanese yields, which have been rising.

On the economic calendar is wholesale trade at 10 a.m. EDT. Kroger and Hovnanian report earnings.

The jump in oil prices to a two-week high Thursday helped fire up the energy sector, which was up 4 percent for the week so far. Oil is up about 7 percent for the week, helped by a record drawdown in U.S. crude supplies as storms disrupted ships bringing foreign oil to U.S. refineries.

Big for the oil market Friday is the 1 p.m. Baker Hughes rig count, which has risen nine of the last 10 weeks. Last week, just one rig was added as oil prices held below $50 per barrel. U.S. West Texas Intermediate futures closed at $47.69 per barrel Thursday, up more than 4.6 percent.

"We've had a lot of whippy trading this week," said Gene McGillian, Tradition Energy market research manager. "If you get another increase or a bigger increase than people are expecting, I would think oil would turn around on the idea that producers are going back to the fields." McGillian and others have pointed to $50 oil as a level that could bring more U.S. production back on line if prices hold.

The big drawdown in U.S. supply also comes in a week where Russia and Saudi Arabia vowed to cooperate. The world's two largest producers would be key to any deal at the meeting of OPEC and non-OPEC producers on the sidelines of an energy conference in Algeria later in the month. Speculation that producers could agree to freeze production has helped lift prices, but many producers are at or near record level of output.

Stocks closed lower Thursday, with the S&P 500 down 4 at 2,181 and the Nasdaq off 23 points at 5,260. The Dow was off 44 at 18,481.

Ari Wald, technical analyst at Oppenheimer, said the market is consolidating after its strong run. Some analysts see a sell-off ahead, and point to the seasonal selling that often goes on in September. But Wald does not see a big move down.

He also points to improved breadth in the market, and an interesting narrowing of the dispersion between sector performance. For instance, the utilities sector is up 20 percent in the last 12 months, as the best performing sector, while the lagging sector, health care, is up 3 percent. The spread between them is 17.

"If you look at the market like that, it's one of the most narrow readings in a decade. I think that speaks to the market rotations that are taking place. This is characteristic of a very healthy market. The S&P 500, there's not one sector really dragging us down," said Wald.