The S&P 500 retreated from a record close it set on Thursday after Ukrainian officials said Russia had launched a "direct invasion" by sending a convoy of trucks across the border.

Investors had been hoping that Yellen would catapult the S&P to the milestone 2,000 level.

While Russia has maintained the trucks contain humanitarian aid, NATO Secretary General Anders Fogh Rasmussen condemned the move as an "alarming build-up." He called on Russia to "stop destabilizing Ukraine."

"People are thinking maybe the convoy sent wasn't in line with an acknowledgment of a Red Cross-led endeavor and was something taken on by Russia unilaterally, which has reestablished a little bit of investor anxiety," said Mark Luschini, chief investment strategist at Janney Montgomery Scott in Philadelphia.

Markets were also pressured by comments from Jackson Hole, Wyoming, where Yellen and European Central Bank President Mario Draghi delivered remarks on the labor market. Yellen did not give a timeline for Fed interest rate hikes, saying that with slack remaining in the labor market, "there is no simple recipe for appropriate policy."

The comments "were more hawkish than what the market was anticipating" because Yellen kept the Fed's ability to hike interest rates sooner than expected on the table, said Karyn Cavanaugh, senior market strategist with Voya Investment Management.

The Dow Jones industrial average fell 38.27 points, or 0.22 percent, to 17,001.22, the S&P 500 ended down 3.97 points, or 0.2 percent, at 1,988.4 and the Nasdaq Composite added 6.45 points, or 0.14 percent, to 4,538.55.

Despite the day's losses, all three major indexes posted gains for the week, with the Dow up 2 percent, the S&P up 1.7 percent and the Nasdaq up 1.6 percent. It was the strongest week of gains for both the Dow and the S&P since April, and the third straight week of gains for all three indexes.

Retailers moved higher, led by a 7.4 percent advance in Ross Stores to $74.37 after the apparel and home fashion retailer posted second-quarter results. The S&P retail index gained 0.6 percent, posting its best week since late February.

The Nasdaq Composite was lifted by Keurig Green Mountain, whose shares soared 13.3 percent to $133.36. The K-cup coffee-pod maker said earlier in the day it had inked a deal to make and sell Kraft Foods Group Inc's coffee.

Peregrine Semiconductor shares jumped 62.9 percent to $12.53 after Murata Electronics North America said it would buy the portion of the chipmaker it does not already own for $12.50 per share.

About 4.1 billion shares traded on all U.S. platforms, according to BATS exchange data, compared with the five-day average of 5.1 billion.