In-Memory Technology Unleashes Real-Time Analytics

Image: infocux Technologies/Flickr

For many years now, the growth and maturity of e-commerce has posed an increasing threat to traditional retailers. While many consumers prefer the experience of physically picking out an item at an in-store retailer, the swing to online is undeniable. According to ShopperTrak, during the holiday shopping season of 2013, in-store retailers saw foot traffic amount to only half of what is was three years prior. Meanwhile, e-commerce keeps growing. So how can brick-and-mortar companies level the playing field and stave off loss of market share from encroaching e-commerce companies?

One way e-commerce vendors have improved the shopping experience is by analyzing the volumes of data they have associated with user logins. By “clustering” similar buying patterns together, they can recommend the consumer products they will most likely want. Until recently, in-store retailers only had dated, traditional ways of competing with this. With the advent of the cheap smartphone, however, retailers now have the ability to treat each consumer that walks in the store as if they’ve just “logged in.” Using low energy technology, an entering consumer can opt-in to being identified. This otherwise simple technological feat opens up a world of possibilities for the retailer to personalize the shopping experience.

When a transaction is made, retailers often collect lots of interesting data. This data can be combined with data being collected from a consumer’s phone in real time as well as RFID sensors on physical items in the store to create a personalized, responsive experience. Using a tablet, store employees can see customer preferences, history and where they are in the store for example. Employees can be alerted if a customer’s size is not available on the floor and must be retrieved. Further, fine-grained buying trends can be easily tracked, like what products are repeatedly left in a changing room, giving the retailers buying power with the manufacturer. Immersive videos can be played when a customer approaches a product area. It’s clear, there are many ways to personalize the in-store experience when data can be analyzed and results delivered in real time.

While the consumer experience is extremely important to compete with e-commerce, there is a host of enabling technology that is necessary to make the consumer experience seamless and performant. A data warehouse technology, in modern times, often Hadoop, is used to store and batch-process historical customer data. More importantly, however, is the middleware used between the data warehouse and the consumer applications. Here is where real-time data streaming in from applications within the store is combined with historical data and analyzed continuously. This technology must scale as more customers enter the store. Traditionally, the only type of technology that meets these needs is in-memory data grid technology. In-memory data grids provide a fast and highly scalable platform for storing data in-memory and running computations over that data. The following diagram illustrates this architecture:

Traditional retailers can use in-memory technology as a catalyst to create a customer experience that is more satisfying than going online. This requires the integration of data warehouse technology like Hadoop, performant, scalable middleware like in-memory data grids, and the sensors within smart phones and RFID tags. Much like online, the in-store experience can now be anchored in the intelligence provided through fast-changing data, but still retain the benefits of traditional shopping. This vision is gaining momentum in retailers around the world and we predict it is well on its way to widespread acceptance.