When to Stop Writing SQL and Start ‘Thinking in Journeys’

A group of analysts were tasked with analyzing customer behavior from a customer journey perspective using a standard analytics approach. They appended unique customer identifiers to every event, loaded the dataset into a massive Spark cluster, executed a complex time-series analysis overnight, debugged, and repeated it until, days later, they finally had success.

Now they needed to go a step further and extract this data to feed machine learning models. The analysts slumped back into their seats, ordered Chinese take-out and prepared to hunker down for the night, hunched over their keyboard and pounding thanklessly on SQL.

Sound familiar?

I am writing this post to tell you that there is another way—a faster, more efficient analytics approach that will give you your life back, while eliminating manual errors.

The Transformational Power of ‘Thinking In Journeys’

Journey analytics is my secret weapon for creating high-quality, visual analysis pulled from a limitless variety of data sources. In 5 minutes I can get whatever data I need for a group of customers exhibiting a highly-specific behavior and analyze it across channels and time with relatively little opportunity for human error.

But let me start at the beginning.

I spent a decade writing SQL as an analyst. Upgrading to SAS Enterprise Miner, a graphical data manipulation tool, was a revelation, but still took hours to run each job, required a dozen QA iterations, and produced a spaghetti-like flowchart of processes which was unintelligible to my colleagues. That was before I exported my data to Excel to actually build my visualizations.

Yet, that was my definition of progress at the time.

Then our enterprise clients began asking a different sort of question, “How can the actions of my customers, across all touchpoints and channels, inform how I design my campaigns, offers, retention tactics, and customer lifetime value strategy?”

Though the terminology wasn’t yet in place, they were looking for what would become known as Customer Journey Analytics. This kind of analysis broke SAS. It broke MySQL. It even broke Hadoop.

The technical explanation behind these failures isn’t complicated, but it’s a story for a different post. Suffice to say, a new technology was needed, and we knew it. This fact didn’t keep our consulting firm (or other consultancies) from pitching journey analytics as if it already existed, but it did keep our analysts from fulfilling that promise.

Now, thinking in journeys has made our customers, and myself, many times more capable as analysts. What if all those past hours staring into SQL had instead been spent prioritizing opportunities and better understanding the business problem? How much more value could we have delivered?

Journey Analytics Represents a New Analytics Approach

When we set out to tackle this challenge, we looked at the full range of existing solutions. Each approached journey analytics in its own way—be it bespoke analytics services or specialized SaaS.

But we wanted a transformational approach to data that embodied a new paradigm for insight discovery, without guardrails, and built around human behavior. In other words, a roadmap for the next generation of analytics. What we found were products and services mostly built around the idea that drawing or visualizing customer journeys was the objective. What we were seeking was a solution where journeys were the language of analysis—a new SQL for the era of artificial intelligence and customer-centricity.

There is a clear understanding among those we work with that journey analytics brings something different. That difference manifests itself as interactive visualizations, blazing fast performance, and analytical power.

Journey analytics is a powerful tool for the rapid discovery of new revenue opportunities.

Accelerated Discovery Leads to Real-world Gains

When it comes to journey analytics—a technically complex, highly specific form of time-series analysis—SQL, database technologies, and even newer solutions like Apache Spark all fall short in terms of both performance and flexibility. Expressing customer behaviors in SQL or other query languages is awkward, difficult to debug, and challenging to communicate and document. Running those queries exerts a large time and resource burden. Sometimes these kinds of queries simply can’t be executed at all.

The real cost of this analytical struggle isn’t time. When a query takes more than 5 minutes to run, an analyst loses the thread of their “conversation” with the data, along with the fleeting intellectual sparks which could have led to the next million-dollar opportunity. When an end-to-end analysis takes days to complete, there often isn’t time to be unsatisfied and try again. There is simply too much pressure to avoid a sunk cost and make the current output work—there are other customers to serve, after all.

Recently, I’ve been working with an analytics leader at an S&P 500 information management company. After he churned out out three real, multi-million-dollar revenue opportunities in the course of training he had this to say about journey analytics:

“Remember that analysis that formerly took us over two weeks to run? Now I can do it in an hour. Our leadership knows the struggles we’ve been through, and this will really resonate with them.”

—Analytics leader at an S&P 500 information management company

These are seasoned analytics professionals, used to having the expertise and data to answer any business question. Using this new analytics approach, they were discovering capabilities that they never knew they were missing.

Now It’s Your Turn

Experienced analysts talk variously about avoiding “the rabbit hole” and “paralysis by analysis.” A purpose-built customer journey analytics tool, designed from the ground up to drive the new generation of customer analytics, all but eliminates these challenges. When a multi-day SQL-coding exercise is reduced to 15 minutes and an hour-long job executes in 3 seconds, there is no “paralysis.”

The rabbit hole is yours to explore. Have at it. Map the entire warren if you like—you’ll be done before lunch! Don’t plan your analysis and check your schedule. Just jump in.

That’s the power of “Thinking in Journeys” and having tools that do the same. When you have the right language to express your analytical thoughts, and the high-performance tools to relieve your time anxiety, access to valuable business of opportunities shifts by orders of magnitude. You don’t need to settle.