From Analysis to Synthesis in Data Science

Jul 06, 2015

The ever burgeoning attention on analytics and big data combined with the ever proliferating phenomenon of smart phones and social media generates great anticipation to the analytically oriented nerd that world domination is now indeed within reach.

However, the analytical approach has been effective in solving relatively low end commercial problems like getting customers to buy more merchandise, speeding up the fulfillment process, reducing transaction costs and as of late, getting an ever increasing number of filtered candidates to fill in job vacancies; just to name some of the more popular applications.

While the scale of the issues addressed is significant, the same cannot be said for the significance of the issues themselves. While the analytical approach is now also being used to solve social problems like crime, sanitation and health care, the resultant outcome in most cases is a downward or upward movement in the success matrices that are used to measure the progress on these fronts. The phenomenon of patting oneself on the back for meeting a numerical target continues to be the norm.

It also almost invariably takes the form of getting people to do more to experience better outcomes (the addition approach) and rarely if ever, prescribes the discontinuation of certain tasks or activities (subtraction approach) to make things better and elegant for the targeted beneficiaries which is a far bigger win-win for both sides.

A synthesis approach, which takes the path of joining the dots (as opposed to splitting the same into smaller and smaller fragments and then combining the ones with similarities) has given the world its greatest breakthrough products. Nothing illustrates this more vividly than the story of how the iPod was created. Sony owned the portable music industry in the form of the Walkman and also owned one of the largest music & entertainment content platforms in the form of Sony Music & Sony Pictures but lost its #1 position in the market since it could not see how these separate dots could be joined.

Meanwhile, Steve Jobs managed to synthesize a connection between digitally streamed music that played on people’s computers and allowed customers to customize their music with single tracks instead of investing in entire albums. The pain point of having to be tethered to their bulky stereos to enjoy their music combined with the ever proliferating trend of listening to music on the move quickly evolved and changed how music was consumed forever. But, Sony remained stuck trying to make incremental gains from their assets through the conventional analysis and research approach.

It was a classic “joining the dots” approach that used both the addition approach (software plus hardware plus a great user experience) and subtraction approach (buying single tracks as opposed to albums/virtual storage vs physical storage) to create a disruptive and game changing product.

BPL Mobile, later acquired by Vodafone, had the opportunity back in the early 2000s to see the effect of this approach in the highly data oriented business of wireless telephony. A high business potential geographic area of India with a lot of heavy users on the post-paid product platform was in the throes of a crisis in terms of mass non-payment of bills, suspensions and subsequent loss of revenues. While diving into the data, the company noticed that the phenomenon repeated itself twice a year at fixed intervals. The classic approach to resolving the issue was to generate more and more precautionary triggers like automated reminders close to payment deadlines. If no payment was made, aggressive in-person visits by collection agents were made with very little improvement which neither prevented the payment lapse from occurring nor successfully collecting the debt after the payment lapse.

This time around they chose not to go by data driven analytical observations alone but also engage in dialogue with customers to understand the causes behind the phenomenon. BPL Mobile discovered that the disruptions were caused by the fact that the town they serviced was predominantly in the business of exporting garments to European & American markets and that the money flowing into the town coincided with the festive seasons in Europe and America. During the high seasons, the customers were flush with cash which they deployed quickly into asset creation leaving them with poor liquidity towards the end of one fulfillment cycle and the start of the next buying cycle.

The challenge was to receive payments when the customers had the funds to meet their annual usage costs and bridge the gap between high seasons. Thus was born a product titled “Bonanza”, where customers could pay a lump sum to cover the cost of the service for the whole year at a substantially discounted rate compared to what they would pay on a monthly basis. The discount being borne out of the cost savings to the company in terms of cost of capital, cost of administering the payment process throughout the year and most importantly, the upsides in revenues while preventing service disruption to the customer. Needless to say the product was a runaway success and soon became an industry standard.

Another example of synthesis approach (subtraction variety) in the telecom industry was solving a problem of poor access and connectivity due to acute network congestion and having to meet revenue goals and higher customer satisfaction scores despite an inability to invest resources to enhance the network capacity. The hunt for a solution began by studying customer behavior during various times of the day to see if there was a possibility to get people to use more Off Peak hours through the incentivizing/de-incentivizing pricing and special benefit mechanisms. This is a classical approach for numerous industries that have high fixed operating costs with big fluctuations in usage during certain fixed phases.

Changing behavior is a very, long drawn out and costly process, and in most cases not an economically worthwhile exercise. The breakthrough was achieved by looking beyond data to identify if there were parts of the service that consumed network time but delivered no consumer value.

The answer lay in understanding the total amount of network time that was consumed when calls went unanswered. A call in most cases goes unanswered when the recipient does not wish to be disturbed or cannot answer their phone. Rarely does a person actually wish to answer a call but is not able to do so quickly enough.

The study revealed 2 things:

The calls rang 8 times before getting automatically disconnected/unanswered

A large % of calls that were answered were within the fourth ring

The company decided to rationalize the number of rings to 6 and released a substantial amount of precious network time from the thousands of calls that went unanswered during the peak hours not only enhancing the customer’s experience but also monetizing the additional resources that this change released.

How does one go about acquiring the capability to synthesize? Are there algorithms capable of bringing this attribute to data science?

My submission on the former is that the first step in this direction is to let go of our obsession with intelligence in the form of our intellectual capabilities - knowledge plus logic plus numerical ability. It starts with focusing inward to understand how our minds function to be able to escape the trap our mechanistic minds subject us to thereby perpetually keeping us in incremental improvement mode.

Google has a very interesting program called, “Search Inside Yourself” which in some ways is an integral part of the training every Googler goes through when developing a high impact skill set. Broaden one’s observation and thinking horizons while still being completely data driven is to take a 3-tiered approach to understanding every business or social problem.

Understanding Moments of Truth - the circumstances and the manner in which the customer is engaging with your product or service or currently trying to solve the problem with what resources are available

Understanding Processes that are involved in managing the moments of truth, distilling and churning the data components at each of these stages, and marrying the analytical observations with what is observed in the real life scenarios.

On the second question, my answer is that I am not qualified to answer it… I am not a data scientist. My limited experience in artificial intelligence tells me that it has made great strides in voice and image recognition but is still a long way off when analyzing emotions. Moreover, as long as we are ruled by our emotions, subjectivity will always be an important component in our lives and thus in our decision making.

I leave you with a very succinct way to grasp the above quote. From an article in the New York Times Magazine interviewing the 25 year old whiz kid opera composer and conductor, Mathew Aucoin, Matthew said “I see Language in Music and Music in Language”. It was termed SYNESTHESIA. A rough parallel for an outstanding Data Scientist would be the ability to see Life as Data and Data as Life.