Tag Archives: retail

One of the most important questions in analytics today is the role for bespoke measurement and analytics versus BI and data visualization tools. Bespoke measurement tools provide end-to-end measurement and analytics around a particular type of problem. Google Analytics, Adobe Analytics, our own DM1 platform are all examples of bespoke measurement solutions. Virtually every industry vertical has them. In health care, there are products like GSI Health and EQ Health that are focused on specific health-care problems. In hospitality, there are solutions like IDeaS and Kriya that focus on revenue management. At the same time, there are a range of powerful, general purpose tools like Tableau, Spotfire, Domo, and Qlik that can do a very broad range of dashboarding, reporting and analytic tasks (and do them very well indeed). It’s always fair game to ask when you’d use one or the other and whether or not a general purpose tool is all you need.

It’s a particularly important question when it comes to in-store location analytics. Digital analytics tools grew up in a market where data collection was largely closed and at a time when traditional BI and Data Viz tools had almost no ability to manage event-level data. So almost every enterprise adopted a digital analytics solution and then, as they found applications for more general-purpose tools, added them to the mix. With in-store tracking, many of the data collection platforms are open (thank god). So it’s possible to directly take data from them.

Particularly for sophisticated analytics teams that have been using tools like Tableau and Qlik for digital and consumer analytics, there is a sense that the combination of a general purpose data viz tool and a powerful statistical analysis tool like R is all they really need for almost any data set. And for the most part, the bespoke analytics solutions that have been available are shockingly limited – making the move to tools like Tableau an easy decision.

But our DM1 platform changes that equation. It doesn’t make it wrong. But I think it makes it only half-right. For any sophisticated analytics shop, using a general purpose data visualization tool and a powerful stats package is still de rigueur. For a variety of reasons, though, adding a bespoke analytics tool like DM1 also makes sense. Here’s why:

Why Users Level of Sophistication Matters

The main issue at stake is whether or not a problem set benefits from bespoke analytics (and, equally germane, whether bespoke tools actually deliver on that potential benefit). Most bespoke analytics tools deliver some combination of table reports and charting. In general, neither of these capabilities are delivered as well as general purpose tools do the job. Even very outstanding tools like Google Analytics don’t stack up to tools like Tableau when it comes to these basic data reporting and visualization tasks. On the other hand, bespoke tools sometimes make it easier to get that basic information – which is why they can be quite a bit better than general purpose tools for less sophisticated users. If you want simple reports that are pre-built and capture important business-specific metrics in ways that make sense right off the bat, then a bespoke tool will likely be better for you. For a reasonably sophisticated analytics team, though, that just doesn’t matter. They don’t need someone else to tell them what’s important. And they certainly don’t have a hard time building reports in tools like Tableau.

So if the only value-add from a bespoke tool is pre-built reports, it’s easy to make the decision. If you need that extra help figuring out what matters, go bespoke. If you don’t, go general purpose.

But that’s not always the only value in bespoke tools.

Why Some Problems Benefit from Bespoke

Every problem set has some unique aspects. But many, many data problems fit within a fairly straightforward set of techniques. Probably the most common are cube-based tabular reporting, time-trended data visualization, and geo-mapping. If your measurement problem is centered around either of the first two elements, then a general purpose tool is going to be hard to beat. They’ve optimized the heck out of this type of reporting and visualization. Geo-mapping is a little more complicated. General purpose tools do a very good job of basic and even moderately sophisticated geo-mapping problems. They are great for putting together basic geo-maps that show overlay data (things like displaying census or purchase data on top of DMAs or zip-codes). They can handle but work less well for tasks that involve more complicated geo-mapping functions like route or area-size optimization. For those kinds of tasks, you’d likely benefit from a dedicated geo-mapping solution.

When it comes to in-store tracking, there are 4 problems that I think derive considerable benefit from bespoke analytics. They are: data quality control, store layout visualization and associated digital planogram maintenance, path analysis, and funnel analysis. I’ll cover each to show what’s at stake and why a bespoke tool can add value.

Data Clean-up and Associate Identification

Raw data streams off store measurement feeds are messy! Well, that’s no surprise. Nearly all raw data feeds have significant clean-up challenges. I’m going to deal with electronic data here, but camera data has similar if slightly different challenges too. Data directly off an electronic feed typically has at least three significant challenges:

Bad Frame Data

Static Device Identification

Associate Device Identification

There are two types of bad frame data: cases where the location is flawed and cases where you get a single measurement. In the first case, you have to decide whether to fix the frame or throw it away. In the second, you have to decide whether a single frame measurement is correct or not. Neither decision is trivial.

Static device identification presents it’s own challenge. It seems like it ought to be trivial. If you get a bunch of pings from the same location you throw it away. Sadly, static devices are never quite static. Blockage and measurement tend to produce some movement in the specific X/Y coordinates reported – so a static device isn’t remotely still. This is a case where our grid system helps tremendously. And we’ve developed algorithms that help us pick out, label and discard static devices.

Associate identification is the most fraught problem. Even if you issue employee devices and provide a table to track them, you’ll almost certainly find that many Associates carry additional devices (yes, even if it’s against policy). If you don’t think that’s true, you’re just not paying attention to the data! You need algorithms to identify devices as Associates and tag that device signature appropriately.

Now all of these problems can be handled in traditional ETL tools. But they are a pain in the ass to get right. And they aren’t problems that you’ll want to try to solve in the data viz solution. So you’re looking at real IT jobs based around some fairly heavy duty ETL. It’s a lot of work. Work that you have to custom pay for. Work that can easily go wrong. Work that you have to stay on top of or risk having garbage data drive bad analysis. In short, it’s one of those problems it’s better to have a vendor tackle.

Store Layout Visualization

The underlying data stream when it comes to in-store tracking is very basic. Each data record contains a timestamp, a device id, and X,Y,Z coordinates. That’s about it. To make this data interesting, you need to map the X,Y,Z coordinates to the store. To do that involves creating (or using) a digital planogram. If you have that, it’s not terribly difficult to load that data into a data viz tool and use it as the basis for aggregation. But it’s not a very flexible or adaptable solution. If you want to break out data differently than in those digital planograms, you’ll have to edit the database by hand. You’ll have to create time-based queries that use the right digital layouts (this is no picnic and will kill the performance of most data viz tools), and you’ll have to build meta-data tables by hand. This is not the kind of stuff that data visualization tools are good at, and trying to use them this way is going to be much harder – especially for a team where a reasonable, shareable workflow is critical.

Contrast that to doing the same tasks in DM1.

DM1 provides a full digital store planogram builder. It allows you create (or modify) digital planograms with a point and click interface. It tracks planograms historically and automatically uses the right one for any given date. It maintains all the meta-data around a digital planogram letting you easily map to multiple hierarchies or across multiple physical dimensions. And it allows you to seamlessly share everything you build.

Once you’ve got those digital planograms, DM1’s reporting is tightly integrated. It’s just seamless to display metrics across every level of metadata right on the digital planogram. What’s more, our grid model makes the translation of individual measurement points into defined areas seamless and repeatable at even fine-grained levels of the store. If you’re relying on pre-built planograms, that’s just not available. And keep in mind that the underlying data is event-based. So if you want to know how many people spent more than a certain amount of time at a particular area of the store, you’ll have to pre-aggregate a bunch of data to use it effectively in a tool like Tableau. Not so in DM1 where every query runs against the event data and the mapping to the digital planogram and subsequent calculation of time spent is done on the fly, in-memory. It’s profoundly more flexible and much, much faster.

Path Analysis

Pathing is one of those tasks that’s very challenging for traditional BI tools. Digital analytics tools often distinguished themselves by their ability to do comprehensive pathing: both in terms of performance (you have to run a lot of detailed data) and visualization (it’s no picnic to visualize the myriad paths that represent real visitor behavior). Adobe Analytics, for example, sports a terrific pathing tool that makes it easy to visualize paths, filter and prune them, and even segment across them. Still, as nice as digital pathing is, a lot of advanced BI teams have found that it’s less useful than you might think. Websites tend to have very high cardinality (lots of pages). That makes for very complex pathing – with tens of thousands or even hundreds of thousands of slightly variant paths adding up to important behaviors. Based on that experience, when we first built DM1, we left pathing on the drawing board. But it turns out that pathing is more limited in a physical space and, because of that, actually more interesting. So our latest DM1 release includes a robust pathing tool based on the types of tools we were used to in digital.

With the path analysis, you start from any place in the store and you can see how people got there and where they went next. Even better, you can keep extending that view by drilling down into subsequent nodes. You can measure simple footpath, or you can look at paths in terms of engagement spots (DM1 has two different metrics that represent increasing levels of engagement) and you can path at any level of the store: section, department, display…whatever.

And, just like the digital analytics tools, you can segment the paths as well. We even show which paths had the highest conversion percentages.

Sure, you could work some SQL wizardry and get at something like this in a general purpose Viz tool. But A) it would be hard. B) it would slow. And C), it wouldn’t look as good or work nearly as well for data exploration.

Funnel Analysis

When I demo DM1, I always wrap-up by showing the funnel visualization. It shows off the platforms ability to do point to point to point analysis on a store and fill in key information along the way. Funnel analysis wraps up a bunch of stuff that’s hard in traditional BI. The visualization is non-standard. The metrics are challenging to calculate, the data is event-driven and can’t be aggregated into easy reporting structures, and effective usage requires the ability to map things like engagement time to any level of meta-data.

In the funnels here, you can see how we can effectively mix levels of engagement: how long people spent at a given meta-data defined area of the store, whether or not they had an interaction, whether they visited (for any amount of time) a totally different area of the store, and then what they purchased. The first funnel describes Section conversion efficiency. The second looks at the cross-over between Mens/Womens areas of the store.

And the third traces the path of shoppers who interacted with Digital Signage. No coding necessary and only minutes to setup.

That’s powerful!

As with path analysis, an analyst can replicate this kind of data with some very complicated SQL or programmatic logic. But it’s damn hard and likely non-performance. It’s also error-prone and difficult to replicate. And, of course, you lose the easy maintainability that DM1’s digital planograms and meta-data provide. What might take days working in low-level tools takes just a few minutes with the Funnel tool in DM1.

Finally, Don’t Forget to Consider the Basic Economics

It usually costs more to get more. But there are times and situations where that’s not necessarily the case. I know of large-scale retailers who purchase in-store tracking data feeds. And the data feed is all they care about since they’re focused on using BI and stats tools. Oddly, though, they often end up paying more than if they purchased DM1 and took our data feed. Odd, because it’s not unusual for that data feed to be sourced by the exact same collection technology but re-sold by a company that’s tacking on a huge markup for the privilege of giving you unprocessed raw data. So the data is identical. Except even that’s not quite right. Because we’ve done a lot of work to clean-up that same data source and when we process it and generate our data feed, the data is cleaner. We throw out bad data points, analyze static and associate devices and separate them, map associate interactions, and map the data to digital planograms. Essentially all for free. And because DM1 doesn’t charge extra for the feed, it’s often cheaper to get DM1 AND feed than just somebody else’s feed. I know. It makes no sense. But it’s true. So even if you bought DM1 and never opened the platform, you’d be saving money and have better data. It would be a shame not to use the software but…it’s really stupid to pay more for demonstrably less of the same thing.

Bottom Line

I have a huge amount of respect for the quality and power of today’s general purpose data visualization tools. You can do almost anything with those tools. And no good analytics team should live without them. But as I once observed to a friend of mine who used Excel for word processing, just because you can do anything in Excel doesn’t mean you should do everything in Excel! In store analytics, there are real reasons why a bespoke analytics package will add value to your analytics toolkit. Will any bespoke solution replace those data viz tools? Nope. Frankly, we don’t want to do that.

I know that DM1’s charting and tabular reporting are no match for what you can do easily in those tools. That’s why DM1 comes complete with a baked-in, no extra charge data feed of the cleaned event-level data and a corresponding visitor-level CRM feed. We want you to use those tools. But as deep analytics practitioners who are fairly expert in those tools, we know there’s some things they don’t make as easy as we’d like. That’s what DM1 is designed to do. It’s been built with a strong eye on what an enterprise analyst (and team) needs that wouldn’t be delivered by an off-the-shelf BI or data viz tool.

We think that’s the right approach for anyone designing a bespoke analytics or reporting package these days. Knowing that we don’t need to replace a tool like Tableau makes it easier for us to concentrate on delivering features and functionality that make a difference.

In my last three posts, I assessed the basic technologies (wifi, camera, etc.) for in-store customer measurement and took a good hard look at the state of the analytics platforms using that measurement. My conclusion? The technologies are challenging but, deployed properly, can work at scale for a reasonable cost. The analytics platforms, on the other hand, have huge gaping holes that seriously limit the ability of analysts to use that data. Our DM1 platform is designed to solve most (I hope all) of those problems. But it’s not worth convincing anyone that DM1 is a better solution unless people get why this whole class of solution is so important.

Over about the same amount of time as those posts, I’ve seen multiple stories on the crisis in mall real-estate, the massive disruption driven in physical retail when eCommerce cross sales thresholds as a percentage of total purchases, and the historical and historically depressing pace of store closings in 2017.

And people get that. The pace of innovation and change in retail has never been as high. Is it high enough? Probably not. But retailers and mall operators are exploring a huge number of paths to find competitive advantage. At a high-level, those paths are obvious and easily understood.

Omni-Channel is Key: You can’t out-compete in pure digital with “he who must not be named”…so your stores have to be a competitive advantage not an anchor. How does that happen? Integration of the digital experience – from desktop to mobile – with the store. Delivering convenience, experience, and personalization in ways that can’t be done in the purely digital realm.

Experience is Everything: If people have to WANT to go to stores (in a line I’ve borrowed from Lee Peterson that I absolutely love), delivering an experience is the bottom line necessary to success. What that experience should be is, obviously, much less clear and much more unique to each business. Is it in-store digital experiences like Oak Labs’ delivers – something that combines a highly-customized digital shopping experience integrated right into the store operation? Is it bringing more and better human elements to the table with personalized clienteling? Is it a fundamentally different mix of retail and experience providers sharing a common environment? It’s all of these and more, of course.

The Store as a Complex Ecosystem: A lot of factors drive the in-store experience. The way the store is laid out. The merchandising. The product itself. Presentations. In-store promotions. Associate placement, density, training and role. The digital environment. Music. Weather. It’s complicated. So changing one factor is never going to be a solution. Retail professionals have both informed and instinctive knowledge of many of these factors. They have years of anecdotal evidence and real data from one-off studies and point-of-sale. What they don’t have is any way to consistently and comprehensively measure the increasingly complex interactions in the ecosystem. And, of course, the more things change, the less we all know. But part of what’s involved in winning in retail is getting better at what makes the store a store. Better inventory management. Better presentation. Better associates and better clienteling strategies. Part of winning in a massively disrupted environment is just being really good at what you do.

The Store in an Integrated Environment: Physical synergies exist in a way that online synergies don’t. In the friction free world of the internet, there’s precious little reason to embed one web site inside another. But in the physical world, it can be a godsend to have a coffee bar inside the store while my daughters shop! Taking advantage of those synergies may mean blending different levels of retail (craft shows, farmers markets) with traditional retail, integrating experiences (climbing walls, VR movies) or taking advantage of otherwise unusable real-estate to create traffic draws (museums, shared return centers).

In one sense, all of these things are obvious. But none of them are a strategy. They’re just words that point in a general direction to real decisions that people have to make around changes that turn out to be really hard and complex. That’s where analytics comes in and that’s why customer journey measurement is critically important right now.

Because nobody knows A) The right ways to actually solve these problems and B) How well the things they’re trying to do are actually working.

Think about it. In the past, Point of Sale data was the ultimate “scoreboard” metric in retail and traffic was the equivalent for malls. It’s all that really mattered and it was enough to make most optimization decisions. Now, look at the strategies I just enumerated: omni-channel, delivering experience, optimizing the ecosystem and integrating broader environments…

Point-of-Sale and traffic measure any of that?

Not really. And certainly, they don’t measure it well enough to drive optimization and tuning.

So if you’re feverishly building new stores, designing new store experiences, buying into cutting edge digital integrations, or betting the farm on new uses for your real-estate, wouldn’t it be nice to have a way to tell if what you’re trying is actually working? And a way to make it work better since getting these innovative, complex things right the first time isn’t going to happen?

This is the bottom line: these days in retail, nobody needs to invest in customer measurement. After all, there’s a perfectly good alternative that just takes a little bit longer.

It’s called natural selection. And the answers it gives are depressingly final.

I didn’t start Digital Mortar because I was impressed with the quality of the reporting and analytics platforms in the in-store customer tracking space. I didn’t look at this industry and say to myself, “Wow – here’s a bunch of great platforms that are meeting the fundamental needs in the space at an enterprise level.” Building good analytics software is hard. And while I’ve seen great examples of SaaS analytics platforms in the digital space, solutions like Adobe and Google Analytics took many years to reach a mature and satisfying form. Ten years ago, GA was a toy and Adobe (Omniture SiteCatalyst at the time) managed to be both confusing and deeply under-powered analytically. In our previous life as consultants, we had the opportunity to use the current generation of in-store customer journey measurement tools. That hands-on experience convinced me that this data is invaluable. But it also revealed deep problems with the way in-store measurement is done.

When we started building a new SaaS in-store measurement solution here at Digital Mortar, these are the problems in the technology that we wanted to solve:

Lack of Journey Measurement

Most of today’s in-store measurement systems are setup as, in essence, fancy door counters. They start by having you draw zones in the store. Then they track how many people enter each zone and how long they spend there (dwell time).

This just sucks.

It’s like the early days of digital analytics when all of our tracking was focused on the page view. We kept counting pages and thinking it meant something. Till we finally realized that it’s customers we need to understand, not pages. With zone counting, you can’t answer the questions that matter. What did customers look at first? What else did customers look at when they shopped for something specific? Did customers interact with associates? Did those interactions drive sales? Did customer engagement in an area actually drive sales? Which parts of the store were most and least efficient? Does that efficiency vary by customer type?

If you’re not asking and answering questions about customers, you’re not doing serious measurement. Measurement that can’t track the customer journey across zones just doesn’t cut it. Which brings me to…

Lack of Segmentation

My book, Measuring the Digital World, is an extended argument for the central role of behavioral segmentation in doing customer analytics. Customer demographics and relationship variables are useful. But behavior – what customers care about right now – will nearly always be more important. If you’re trying to craft better omni-channel experiences, drive integrated marketing, or optimize associate interactions, you must focus on behavioral segmentation. The whole point of in-store customer tracking is to open up a new set of critically important customer behaviors for analysis and use. It’s all about segmentation.

Unfortunately, if you can’t track the customer journey (as per my point above), you can’t segment. It’s just that simple. When a customer is nothing more than a blip in the zone, you have no data for behavioral segmentation. Of course, even if you track the customer journey, segmentation may be deeply limited in analytic tools. You could map the improvement of Adobe or Google Analytics by charting their gradually improving segmentation capabilities. From limited filtering on pre-defined variables to more complex, query-based segmentation to the gradual incorporation of sophisticated segmentation capabilities into the analyst’s workbench.

You can have all the fancy charts and visualizations in the world, but without robust segmentation, customer analytics is crippled.

Lack of Store Context

When I introduce audiences to in-store customer tracking, I often use a slide like this:

The key point is that the basic location data about the customer journey is only meaningful when its mapped to the actual store. If you don’t know WHAT’S THERE, you don’t have interesting data. The failure to incorporate “what’s there” into their reporting isn’t entirely the fault of in-store tracking software. Far too many retailers still rely on poor, paper-based planograms to track store setups. But “what’s there” needs to be a fundamental part of the collection and the reporting. If data isn’t stored, aggregated, trended and reported based on “what’s there”, it just won’t be usable. Which brings me to…

Use of Heatmaps

Heatmaps sure look cool. And, let’s face it, they are specifically designed to tackle the problem of “Store Context” I just talked about. Unfortunately, they don’t work. If you’ve ever tried to describe (or just figure out) how two heat-maps differ, you can understand the problem. Dialog like: “You can see there’s a little more yellow here and this area is a little less red after our test” isn’t going to cut it in a Board presentation. Because heat-maps are continuous, not discrete, you can’t trend them meaningfully. You can’t use them to document specific amounts of change. And you can’t use them to compare customer segments or changed journeys. In fact, as an analyst who’s tried first hand to use them, I can pretty much attest that you can’t actually use heat-maps for much of anything. They are the prettiest and most useless part of in-store customer measurement systems. If heat-maps are the tool you have to solve the problem of store context, you’re doomed.

These four problems cripple most in-store customer journey solutions. It’s incredibly difficult to do good retail analytics when you can’t measure journeys, segment customers, or map your data effectively onto the store. And the ubiquity of heat-maps just makes these problems worse.

But the problems with in-store tracking solutions don’t end here. In my next post, I’ll detail several more critical shortcomings in the way most in-store tracking solutions are designed. Shortcomings that ensure that not only can’t the analyst effectively solve real-world business problems with the tool, but that they can’t get AT THE DATA with any tools that might be able to do better!

Want to know more about how Digital Mortar can drive better store analytics? Drop me a line.

I’m working my way through the broad uses of in-store customer journey optimization. I started with Store Layout and Merchandising optimization – which is really the foundational analytic capability that this type of data provides. Today, I’ll tackle a use that’s nearly as fundamental – optimizing in-store promotions. For those of you from the digital world, you can think of these two applications as parallel to site optimization and digital marketing optimization.

Promotion Planning

In-store promotion planning is one of those constant grinds in the life of retail analysis. You never stop planning promotions and you never get good enough. With PoS data, it’s pretty easy to measure the single most important aspect of a promotion – how much it sold. It can be a lot harder, however, to answer questions about why something worked or, as is often more salient, why something didn’t. In-store measurement can fill in the gaps around performance measurement AND help develop new promotion and display strategies.

With in-store journey measurement, you can track how and whether a promotion shifted behavior. Did a promotion steer visitors to a section? Did it keep them there longer? Did it drive key milestones like staff interaction or dressing room decisions? With only PoS data, you can easily misunderstand what drove a promotion’s apparent effectiveness. Almost as important, in-store journey measurement provides unique insight into how a promotion cannibalized shopping behaviors and generated new opportunities. When you change navigation patterns in the store, you ALWAYS cannibalize some behaviors and you nearly always disadvantage some sections/products. You also create new opportunities and traffic corridors that might present additional optimization or promotion opportunities. Understanding how cannibalization and redirection worked and whether or not their impact outweighed the promotion benefits is essential to developing sound long term strategies.

And it’s not all about the customer. In digital analytics, we didn’t have to worry much about compliance issues. What you pushed to the website is what was on the website. With dozens, hundreds or thousands of stores to manage, though, pushing content and making sure it’s consistent and correctly deployed is no joke. In-store customer journey measurement provides a strong behavioral compliance check. When a promotion drives specific patterns of behavior, it’s easy to see which stores are roughly following the pattern and which aren’t – given you near real-time feedback on potential compliance issues.

Questions you can Answer

Why did a promotion work better or worse than expected?

How did promotions localize and were there stores that didn’t “play along”?

How much opportunity did promotions have to influence shopping?

How successful were shoppers who were exposed to the promotion?

Did the promotion create new “impulse” opportunities?

Did the promotion cannibalize other areas/products and to what extent?

For a potential promotion, what are they placement areas that will drive exposure to the right shopping segments?

Were there stores that didn’t deploy or correctly implement a promotion?

In the last few months I’ve been spending quite a bit of time thinking about the challenges in physical retail – stores. I’m going to be talking much more about that in the months to come, but thinking about the challenges in physical retail and whether and to what extent digital techniques might help, I’ve also had to think about why digital retail has evolved the way it has.

There’s no doubt that digital has disrupted and hurt traditional retail. But it’s a mistake to attribute that solely to advantages inherent in digital. After all, if it was just a matter of digital being superior to B&M, then Borders should have been fine moving online. That didn’t work out so well.

In fact, one of the most interesting aspects of our digital world is how a perfect leveling of the playing field has produced such a strong tendency to natural monopoly. This isn’t just about retail. In most of the key areas of internet – from retail to video streaming to music to search to ride summoning, we’ve seen an extraordinary tendency toward massive consolidation around a single leader.

It’s not exactly what most of us expected. By eliminating most barriers to entry, creating frictionless geographies, and creating technology environments that scale seamlessly to almost any size, the digital world has removed many of the traditional bastions of monopoly. Old-world monopolies used to spring from cases where scale precluded competition. If, for example, you owned the pipes that carried gas to homes or the wires that carried electricity, it was incredibly hard for anyone else to compete.

In today’s world, that kind of ownership has mostly vanished. You could argue that if you own search you own the pipes to the Web. But the analogy doesn’t hold. It doesn’t hold because anybody can create a competing search system at any time and every single internet user can have instant access to it. It doesn’t hold because there are multiple ways to pipe through the internet besides search. And it doesn’t hold because there really are no physical barriers to building or deploying that alternative search system.

So it wouldn’t be unreasonable to expect the digital world to have morphed into a wild west of tiny artisanal companies with meteoric rises, equally sudden collapses, and constant, ubiquitous competition. Mostly, though, that’s not the way it looks at all. It looks as if monopoly, despite the absence of physical barriers, is actually a more powerful tendency in the digital world than the physical world.

It’s not that hard to understand why things have gone this way. Natural monopolies around things like electricity delivery occurred because of the immense friction involved in setting up the delivery system. Economies of scale were absolutely decisive in such situations. But most traditional markets are resilient to natural monopoly because of fundamental facts of the physical world that worked AGAINST too much scale. In the physical world, it makes perfect sense to have gas stations on the opposite side of a street. And it’s quite likely that two such stations can not only co-exist but thrive despite their close proximity. After all, it’s a pain to cross the street when you want to get gas. I may prefer Whole Foods to Safeway or vice versa. But I often go the grocery store that’s closest to me regardless of brand. And when I lived in San Francisco I bought most of my Diet Coke and impulse snacks at the corner store up my block. No, it wasn’t nice and it wasn’t cheap. But it sure was close. I may like Sol Food in San Rafael better than Los Moles, but so do a lot of other people – and I hate standing in line.

The natural friction that the physical world carries in terms of geographic convenience and capacity help ensure that countless niches for delivery exist. Like my old corner store, in the physical world, you can o be worse at everything except location and still thrive.

That doesn’t happen in the digital world.

It turns out – and I guess this should be no surprise – that in a frictionless world, any small advantage can be decisive. A grocery has to be a LOT better than its competitors to get me to drive an extra 10 minutes. But online, the best grocery is always just a few milliseconds away.

It doesn’t have to be a lot better. In fact, the difference can be incredibly tiny. Absent friction, the size of the advantage is no longer that meaningful. The digital world can make even tiny advantages decisive.

So why doesn’t every aspect of the digital world turn into a monopoly?

The answer lies in segmentation. A very small advantage may be decisive in the digital world. But it’s hard to have an advantage to EVERYONE.

In areas like news and entertainment, for example, it’s impossible to produce content that is better for everyone. Age, education, interest, background, geography and countless other factors create an infinity of micro-fractures. Not only is the content itself differentiated, but it’s creation is almost equally fractured. A.O. Scott could no more produce a version of Real Housewives than Andy Cohen could write a NY Times film review.

Content creation turns out to be friction-full in a way that was somewhat obscured by the old limitations in distribution. In fact, it appears that the market for segmented content and the ability of content to create barriers to consolidation is almost limitless. That’s why there’s almost nothing so important to becoming a good digital company than content creation. It’s the best way there is to guard your marketspace.

All this suggests that there are two paths to success in the digital world. One path involves scale and the other segmentation. They aren’t mutually exclusive and the companies that do both well are formidable indeed.

It’s only a little more than a month till the Digital Analytics Hub in Monterey and a chance to talk all things digital – both practical and philosophical. After all, there is no monopoly on great conversation. Looking forward to talking deep analytics, natural monopolies, digital transformation and digital advantage!

People have struggled with this (big) data provider model but Factual feels like it’s found a real (and valuable) niche. Would love to see more of this grow since external data is a huge miss in most big data systems.

Targeted VoC is a powerful (and totally neglected) tool for personalization. Facebook’s experience is entirely relevant to ANY content producer. I don’t know if I can take credit for this, but I suggested this to folks at Facebook a couple of years back!

An interesting discussion of the problems in identifying “likely” voters and the benefits of behavioral data integration. Food for thought in the enterprise world as well where the equivalent is often possible but rarely done.