Agilisium architected, designed and delivered an elastically scalable Cloud-based Analytics-ready Big Data solution with AWS S3 Data Lake as the single source of truth

The client is one of the world’s leading biotechnology company, with presence in 100+ markets globally, was looking for ways to maximize impact of their sales & marketing efforts.

The lack of a single source of truth, quality data and ad hoc manual reporting processes undermined top management’s visibility of integrated insights on sales, sales rep interactions, marketing reach, brand performance, market share, and territory management. Understandably, the client wanted to align information that has hitherto been in silos, to gain a 360-degree product movement view, to optimize sales planning and gain competitive edge.

The Challenge

The major setback for the Client was, discrete data, & also not in readily available & easily consumable fashion. The established markets were lacking single source of truth & the growing market was lacking proper data collection mechanism. The key challenge is to offer a Cloud based scalable Reporting & Analytics’ environment that can host established & new market data to provide 360-degree view of product, customer & Geography behaviour.

The existing system architecture was discrete and led to siloed data sources. The lack of strong governance undermined scope management and integrated insights generation. The key challenge was to offer a scalable Big Data Analytics solution that supports business objectives, while catalysing enablement of strong governance frameworks at enterprise level.

Our Solution

A Cloud One strategy to progressively transition to a globally accessible and scalable Big Data Analytics platform was devised. Agilisium architected, designed and delivered an elastically scalable Cloud-based Analytics-ready Big Data solution with –

The framework sends automated email notification to data stewards at each step, whenever a data quality issue is encountered. The stewards view error records through Tableau, upload and approve corrected records for loading.

User and country-based security through IAM and row-level security is applied in both S3 Data Lake and AWS Redshift Analytics Data Warehouse.

The data was processed and loaded in multilevel/layered as Raw/ Integrated Granular level data/ aggregated data with Business rules applied in AWS Redshift, to provide Multi-tier sematic layer for Tableau.

Key Highlights

Technologies – AWS S3, AWS Redshift, Snaplogic, Tableau

Team size – 1 (Onsite), 3 (Offshore)

Data Size – About 50 MN records for 3 global markets

Project Duration – 5 months (ongoing)

Delivery model – Hybrid

How We Worked Together

Post a maturity assessment, Agilisium devised and enacted strong governance frameworks at executive, program and operational levels with scheduled check point review meetings, to close current gaps identified in scope management.

Scrum calls twice daily ensured that application owners were apprised about progress made at operational level. While monthly milestone review meetings aligned priorities at program level, the quarterly executive steering committee meeting clearly set the engagement priorities at executive level.

To ensure a smooth transition of new solutions, a 1-week workshop and demo on S3, Redshift and Tableau were given to the client’s business and technical teams.

Results and Benefits

Holistic 360-degree view – 360-degree view on product movement and sales progression is now available with reduced dependency on IT.

Quicker Insights – With access to data at lowest grain, business leaders answer their new questions with fewer clicks through self-service BI tool.

2x faster territory transition – Metadata has driven generic framework helped the transition of new territories in half the time than planned.

Advanced Analytics Ready – The new architecture supports scalable Advanced Analytics, at the speed of thought.