Despite strong sales early on, factors including the looming "fiscal cliff" have put a damper on this year's holiday season as consumers struggle with shopping indecision.

So far, retail sales are up just 0.7 percent, which is the lowest growth since the financial crisis in 2008, according to data from MasterCard Advisors SpendingPulse,

Although Michael Brown, a partner at the management consulting firm, A.T. Kearney, said about 95 percent of sales are already in the books, he thinks there are still some strong days ahead that could move the season one way or another. Brown said most of the consensus estimates for holiday season growth are in the range of 2.5 to 3 percent. He added that it will likely be closer to the latter.

"There are a couple of big days still coming," Brown said. "I think this weekend is a big weekend as people redeem gift cards, go into the stores, exchange items for what they're looking for, hopefully get that add-on sale and people start to take advantage of the clearance sales."

Although this year's holiday shopping season began with a strong start, Hurricane Sandy's destruction along with economic uncertainty put a damper on spending during the weeks between Black Friday and Christmas, he said. The lack of a resolution to the fiscal cliff — when automatic tax hikes and spending cuts are set to occur on Jan. 1 — has also affected consumers' spending patterns.

"What you've got is a consumer who wants to go out and make a purchase, but they don't know — Do I buy the $300 item because I'm going to get a tax break or do I buy the $250 item because I'm not? — and they're just waiting," Brown said.

Paper Boat Creative | Lifesize | Getty Images

Once Washington does come up with a resolution, people will start shopping again, Brown predicted in an interview with CNBC's "Squawk Box."

This year also saw Cyber Monday smash the record for the biggest online shopping day ever as sales rose 30 percent over last year, according to data from IBM.

"The consumer is shopping online, and I think that puts a damper on sales also because the online shopping experience is very targeted," Brown said. "It's very mission focused."

This new laser-like focus means consumers are not venturing as often into brick-and-mortar stores, where aisle browsing often yields impulse purchases.

But Brown cautioned against thinking that retailers' drive to sell more merchandise online is a mistake.

"It was inevitable but I think what they have to focus on this year is driving them back to the store," he added. To lure consumers back, Brown suggested hosting more in-store events and entertainment along with making more of an effort to create relationships with consumers, such as the Macy's campaign that allows children to write to Santa.