August 3, 2011

Subscribe

Big Data, Analytics and KPIs in E-commerce and Retail Industry

How to create unique and effective Digital Experiences that impact probability of purchase or likelihood of return.

What offers might result in higher “take rates”

The change in consumer behavior and expectations that e-commerce, mobile and social media are causing is hugely significant – big data and predictive analytics will separate brand/retail winners from losers. This won’t happen overnight but the transformation is for real.

Retail is increasingly is looking like a barbell – a brand oriented cluster at the high-end, a very thin middle, and a price sensitive cluster at the low end. The consumerization of technology is putting more downward pricing pressure in an already competitive “middle” retail environment. The squeeze is coming from e-commerce and new “point, scan and analyze” technologies that give shoppers decision making tools — powerful pricing, promotion and product information, often in real-time. Applications in iPhones and Droid, like Red Laser can scan barcodes and provide immediate price, product and cross-retailer comparisons. They can even point you to the nearest retailer who can give you free shipping (total cost of purchase optimization). This will lead to further margin erosion for retailers that compete based on price (a sizable chunk of the market in the U.S, Europe and Asia).

Data analytics is not new for retailers. Point of sale transactional data obtained from bar-codes first appeared in 1970s. A pack of Wrigley’s chewing gum was the first item scanned using Universal Product Code (UPC) in a Marsh Supermarket in Troy, Ohio in 1974. Since then, retailers have been applying analytics to get even smarter and speedup the entire industry value chain.

More recent use cases of retail analytics include:

Corporate Express is using its market basket application to track the baskets of all of its online customers, and can identify which SKUs are most profitable or poor performers when purchased in combinations.

Netflix is a heavy user of big data… The company issued a $1M analytics challenge in October 2006: come up with recommendation software that could do a better job predicting the movies customers would like than Netflix’s in-house software, Cinematch. To qualify for the prize, entries had to be at least 10 percent better than Cinematch. (for more detail see notes at end).

BestBuy had a clickstream analysis problem — 3.5 billion records, 71 million unique cookies, 1.7 million targeted ads required per day. How to make sense of this data? They hired Razorfish to implement a analytic solution on Amazon Web Services and Elastic MapReduce. Solution was a 100 node cluster on demand; processing time was reduced from 2+ days to 8 hours.

In a business with razor-thin margins, small, mid and large retailers are constantly looking for the performance improvement edge. Retailers are racing to become better at slicing and dicing reams of data they collect from multiple channels – online, stores, call center, catalogs and smartphones. They are using business analytics to get insight into retail KPIs (outcome measures) or Metrics (operational activities that serve as leading or lagging indicators). The goal is to drive significant topline and bottomline improvements through improved merchandising decisions, more timely responses to information requests, cost reduction initiatives, enhanced employee productivity, and better service to customers.

Types of Big Data Analytics in Retail

We analyzed the typical retailers and found some interesting patterns of how raw data is being converted into signals and actions. Most of the Retail Analytics and Big Data retail tend to cluster around the following areas:

It’s important to distinguish between retail KPIs and metrics. We view KPIs as clusters of metrics that in aggregate provide a report card. Metrics on the other hand are simply snapshots of an activity. Think of KPIs as overall health vs. metrics as taking temperature or measuring shortness of breadth.

Customer Analytics and KPIs

Objective: Understand Your Most Valuable Customers. Target Them to Maximize Profits and Loyalty

Every retailer is focused on understanding and leveraging all customer touch points, from granular in-store transactional data to online store and web search information to mobile data, in order to most effectively target customers, improve repeat purchase, drive cross-sell. All to maximize profits and customer loyalty.

Once you understand your customer at this level, you can align effective tactics to tailored customer segments to surround them with a holistic shopping experience. Enable business users in marketing and merchandising to seamlessly integrate insight with promotion planning, weekly flyers, planograms, store assortments, and more. Boost offer performance. Drive larger baskets. End goal – increase sales, customer profitability, and customer loyalty.

Merchandizing KPIs

Objective: Significantly reduce costs, eliminate the expense of stock-outs and overstocks, and make powerful, rapid decisions

Retailers face immense pressure to balance inventory levels with in-stock positions while maintaining assortments that are relevant and fresh. Buyers and merchandise managers need to be constantly on-the-go either in team meetings, traveling overseas for a buying trip, or working hand-in-hand with vendors.

Whether at their desk or on-the-go, this job requires mobility and constant connectivity to respond to consumer demand, seasonal trends, and fashion styles, all to ensure that customers get the right products.

Merchandise Analytics and KPIs — Eliminate Stockouts and Overstocks

Department Contribution

Hot Item Report

In Stock %

Inventory Turns

Lineal Feet

Mark Down %

Pull-Through

Reallocations

Seasonal Buying

Sell-Through

Weeks of Supply

The Merchandise Analysis scenarios are designed to improve inventory decisions. Users can quickly accelerate shipments by evaluating top-selling products, make markdown decisions based on seasonal sell-through, cancel shipments for bottom-selling products, and communicate more effectively with vendors.

Store Operations Analytics and KPIs

Objective: Keep Store Managers on the Selling Floor, Not Behind a Desk. Give Store Operations the Right Information at the Right Time to Make the Right Decisions.

Store managers are more effective when they are out on the selling floor, not in a back office. The old paradigm of paper reporting or even web/desktop is now becoming obsolete. Mobility empowers field managers in stores to make near real-time decisions regarding inventory, promotions, pricing, staffing, and store design.

Store managers can now run daily analyses of store performance where they can aggregate by Store sales, SKU sales and sales per employee. In some advanced retailers, the data is refreshed in real-time so they can deal with just-in-time replenishment more cleverly.

Mobile store operations reporting keeps the store managers where they should be: in the store, on the selling floor. Store managers can see, at regular intervals throughout the day, how their sales are performing compared to last year, giving them the information and tools to improve performance.

Area

Type of Analytical Insights

Store Operations Analytics and KPIs

Asset Turnover

Competitor Stores

Comp Store Sales

Front Store Sales

Inventory Turnover

Labor Cost Analysis

Reallocation / DSD

Register Usage Analysis

ROA

Sales & Margin

Sales / Square Foot

Store traffic information delivers detail on a store’s performance: Units per Transaction, Average Transactions (Units and $), Conversion Rates, and Sales per Payroll Hour are all provided in the mobile BI app.

Relevant, filtered information delivered when and where it is needed helps store personnel identify anomalies in their operations and drive immediate action in the areas of labor scheduling, customer service, merchandise display, promotional signage, potential stock shortfalls caused by local market conditions, and more.

Vendor and SKU Management Scorecards and KPIs

Managing vendors in today’s environment requires you to know what is happening in the supply chain real-time, enabling you to make rapid decisions to optimize performance and to strengthen your negotiations on price and quality.

Area

Type of Analytical Insights

Category Management KPIs

Assortment Optimization

Category Contribution

Category Scorecards

Item Definitions

Market Comparison

Pricing Modules

Role Analysis

Season Changeover

Share of Shelf

SKU Rationalization

Vendor Performance

A comprehensive Vendor Scorecard provides the right information to take the right action at the right time. A scorecard helps jointly improve performance by providing a common view of the following:

How are my products / categories performing?

Where do I have issues with product sales, returns, or excess inventory?

Marketing Analytics and KPIs

Marketing is going through tremendous changes. As more customers move online and social becomes a big factor in how people are influenced to make a purchase, new techniques are emerging around cross-selling, location-based marketing, in-store behavior analysis, customer micro-segmentation, sentiment analysis and enhanced multichannel consumer experience.

Retailers are using big data to integrate marketing promotions, location data and differential pricing. Promotions via large-scale e-mail targeting (many retailers have 25 to 50 million household lists) is still a big challenge for many retailers. Open rates on targeted e-mails with custom catalogs with tailored personalized storefronts are 10 times more effective than those that are not targeted.

Another interesting emerging area of retail big data is sentiment analytics. Combination analytics from integrating profiles, peer sentiment, recommendations and comments on Twitter, Yelp and Facebook are playing a bigger role in influence and purchase decisions. As a result, there is a race among market departments to leverage the voluminous streams of data in social media to drive effective promotions or messages at various touch-points. Social discussion of brands, products and services, and endorsement of them, is very influential. Consumers have always been influenced by the opinions of others, and ignoring a channel that can influence so many consumers so quickly would be unwise.

Competition for the 16-35 age group customers is intense. Emerging social marketing services like Foursquare are changing the dynamics of real-time customer interaction in that age group. Foursquare is a web and mobile application that allows registered users to connect with friends and publish their location. Users “check-in” at venues using a mobile website, text messaging or a device-specific application by clicking from a list of venues. Each check-in awards the user points and sometimes “badges”. As of June 2011, the company reported it had 10 million registered users. Foursquare is trying to become a sticky portal by moving from social connection to promotions/coupons to even payments.

A list of classic marketing KPIs is provided below.

Area

Type of Analytical Insights

Marketing Analysis KPIs

Attrition and Fading

Ad Blocks

Channel Share

Coupon Distribution

End-Cap Efficiency

Feature vs. Display

Marketing Areas

Market Share

Price Points

Promo Lift

Seasonal Events

Site visitors, Click-thrus

Returns, Fraud and Loss Prevention Analytics and KPIs

Ease of returns has always correlated well to customer satisfaction. Thanks to Amazon Prime or Zappos’s 365 day return policy the dynamics of the industry are shifting. Customers love this policy and are eager to migrate to vendors that provide flexibility. But the challenge is predicting return trends by product and sku; and keeping necessary reserves.

In a low margin industry, loss prevention is often a big focus. Retail theft jumped by $3.6 billion to $37.1 billion in 2010, according to National Retail Federation. More goods (44 percent) were swiped by employees, often working with organized groups of thieves, than shoplifters, which accounted for $12 billion, or 33 percent of lost goods. Paperwork errors or vendor fraud accounted for $6.8 billion, or 18 percent.

A list of classic Loss Prevention KPIs is provided below.

Area

Type of Analytical Insights

Loss Prevention KPIs

Backorders

Cancels

Distribution Shrink

Hot Customer

Hot Staff

Hot Stores

Markdowns

Natural Losses

Return Rates

Shrink Analysis

Vendor Rationalization

Bottomline

The use of IT and digital data in the retail industry has been instrumental in boosting the profitability and productivity of players like Walmart, Tesco, Amazon.com for years. In the coming years, the increased adoption of big data has the potential to increase sector-wide productivity.

In an industry with notoriously tight margins (typically 2-6% range), retailers are facing a set of challenges that require a new set of solutions to address them. Margin declines can no longer be offset by improvements in selling, general, and administrative (SG&A) costs and reduced inventories.

Retailers will need to selectively employ a set of levers that they have utilized in the past to manage costs in the short-term, and a series of more-advanced levers to build a sustainable competitive advantage.

3. Geographic retailing is gaining a foothold with better tools. Placecast, a provider of location-based marketing, claims that 50% of users have made purchases as a result of location-based ads (targeted ads combined with preference, profile and location)

4. Williams-Sonoma, all things kitchen and cooking, has a database of 60M households tracking variables like income, number of children, housing values, etc. They leverage these variables in e-mail targeting programs.

5. The Netflix movie recommendation contest (blending of different statistical and machine-learning techniques) has been widely followed because its crowdsourcing lessons could extend beyond improving movie picks. The researchers from around the world were grappling with a huge data set — 100+ million movie ratings — and the challenges of large-scale predictive modeling.

6. Interestingly several people think that “what your friends thought” feature to be extremely accurate in predicting and suggesting movies…more than the recommendation feature.

7. Netflix announced a second recommendation contest. Contestants were asked to model individuals’ “taste profiles,” leveraging demographic and behavioral data. The data set — 100 million entries will include information about renters’ ages, gender, ZIP codes, genre ratings and previously chosen movies. Unlike the first challenge, the contest will have no specific accuracy target. $500,000 will be awarded to the team in the lead after six months, and $500,000 to the leader after 18 months. This contest was cancelled in May 2010 after a legal challenge that it breached customer privacy with the first contest.

8. Typical problems retailers face:

Low repeat purchase

Fading and attrition

No cross-selling or low expansion into new categories

Pricing inefficiencies

Post-promotion inventory gluts

9. Healthcare Analytics Challenge — California physicians group Heritage Provider Network Inc. is offering $3 million to any person or firm who develops the best model to predict how many days a patient is likely to spend in the hospital in a year’s time. Contestants will receive “anonymized” insurance-claims data to create their models. The goal is to reduce the number of hospital visits, by identifying patients who could benefit from services such as home nurse visits.

10. As one our comments noted…. It’s important to distinguish between retail KPIs and metrics. We view KPIs as clusters of metrics that in aggregate provide a report card. Metrics on the other hand are simply snapshots of an activity. Think of KPIs as overall health vs. metrics as taking temperature or measuring shortness of breadth. KPIs are like latent variables statistics (as opposed to observable variables). KPIs are variables that are not directly observed but are rather inferred (through a mathematical model) from other variables that are observed (directly measured).

Good detailed read however, one of the significant department i.e. IT’s ( Information Technology) KPI is missing. If the same be included into it, it would be more insightful for the reader (Top Management).

Defining Business Analytics

What is Business Analytics? Business Analytics is the intersection of business and technology, offering new opportunities for a competitive advantage. Business analytics unlocks the predictive potential of data analysis to improve financial performance, strategic management, and operational efficiency.

What is BI? BI is the "computer-based techniques used in spotting, digging-out, and analyzing 'hard' business data, such as sales revenue by products or departments or associated costs and incomes. Objectives of BI implementations include (1) understanding of a firm's internal and external strengths and weaknesses, (2) understanding of the relationship between different data for better decision making, (3) detection of opportunities for innovation, and (4) cost reduction and optimal deployment of resources." (Business Dictionary). Most widely used BI tool is Microsoft Excel.
-----------------------------
What is Big Data? Big data refer to data scenarios that grow so large (petabytes and more) that they become awkward to work with using traditional database management tools. The challenge stems from data volume + flow velocity + noise to signal conversion. Big data is spawning new tools that are mix of significant processing power, parallelism and statistical, machine learning, or pattern recognition techniques
----------------------------

Corporate performance management software and performance management concepts, such as the balanced scorecard, enable organizations to measure business results and track their progress against business goals in order to improve financial performance.
-----------------------------

Data visualization tools, include mashups, executive dashboards, performance scorecards and other data visualization technology, is becoming a major category.
-----------------------------

BI platforms provide a range of capabilities for building analytical applications. Examples are Oracle OBIEE, SAP Business Objects 4.0. There are many choices and combinations of BI platforms, capabilities and use cases as well as many emerging BI technologies such as in memory analytics, interactive visualization and BI integrated search. The idea of standardizing on one supplier for all of one’s BI capabilities is difficult to do. Increasingly, standardization and more about managing a portfolio of tools used for a set of capabilities and use cases.
----------------------------
Data integration tools and architectures in support of BI continue to evolve. Extract-Transfer-Load (ETL) tools make up a big segment of this category in addition to data mapping tools. Organizations must now support a range of delivery styles, latencies, and formats.
----------------------------
BI is about "sense and respond." Analytics is about "anticipate and shape" models.

About

Business Analytics 3.0 blog is meant for decision makers and managers who are trying to make sense of the rapidly changing technology landscape and build next generation solutions. It is aimed at helping business decision makers navigate the "Raw Data -> Aggregate Data -> Intelligence -> Insight -> Decisions" chain.