‘Big data’ is still a big mystery

While 80% of retailers say they’ve heard of “big data,” only 47% say they understand how to apply it to their business, according to a survey by retail research firm Edgell Knowledge Network.

The market research firm arrived at its findings by interviewing 75 North American retail executives. It also studied the data analysis practices of 100 other retailers and big data service providers to create its summer 2012 “Big Data in Retail” report.

Edgell defines big data as a set of data that has become too large and complex for standard computing tools to capture, store and manage. For example, the billions of Likes, comments and other interactions on social media networks alone require many servers and sophisticated software to process all that data. Big data takes into account two types of data: structured and unstructured. Structured data refers to data that fit easily into existing databases, such as transactions, conversions or other numeric values. Unstructured data is more amorphous—like the text in a Tweet or the content of an audio or video file—and needs to be somehow standardized and ordered before it can be analyzed.

57% of survey respondents say they have or are planning a big data strategy for their business and 30% say they’ve already completed a big data project. Online retailers have an advantage when it comes to acting on big data, says Edgell research director Gaurav Pant. That's because e-retailers often are more nimble at changing things on the fly—for example, product pricing or the terms of loyalty programs.

46% of retailers in the survey say the volume of data with which they must contend is the biggest challenge in dealing with big data, 34% say managing data variety—both structured and unstructured data—is the top challenge and 20% say handling the data’s velocity, or the frequency at which it is generated and captured, is hardest.

Based on the report, Pant advises retailers to focus on three key areas where big data analytics can be applied to their businesses. The first is to optimize pricing with computer algorithms that compare the cost of items at many e-commerce sites, then automatically adjust one’s own prices to be the lowest. The second area is micro-segmentation, which refers to creating new customer groups based on previously untapped categories—for example, the followers of an influential Pinterest user. The last is marketing, which can be better targeted and personalized the more data a retailer uses to serve up products and advertising to individual consumers.

These strategies aren’t just for big businesses, Pant says. Advances in technology, especially those using a software-as-a-service model—for which retailers pay a monthly subscription fee to access software hosted on vendors’ servers over the Internet without needing to invest in hardware and software licenses themselves—is lowering the price and staffing requirements for all retailers to do big data analyses. Outdoor gear and apparel retailer Moosejaw Mountaineering is a leader among small retailers using big data, especially from social media, to improve business, he says.

The growing speed and amount of access to consumer information will soon spark a widespread organizational change for all business executives in retail, says Dave Weinand, vice president of retail at Edgell Knowledge Network. “I hear this all the time from retailers I talk to: It’s really hard to find people who can analyze this data in a quantifiable way,” Weinand says. And those people need to work on the marketing and management sides of the business too, not just in information technology, he adds.