The Self-Service Data Delivery Integration Platform for Next Generation Companies

Not long ago I presented at the MuleSoft TopMule Meetup ’14 in San Diego, CA, on an initiative I’m running at salesforce.com called Free The Data: Transforming The Way The Business Connects To Data. With approximately 400 MuleSoft employees in attendance, MuleSoft has doubled its size in just one year. It’s a testament to the industry demand for their integration and API solutions. My talk centered around how we are aggregating, exposing and simplifying access to internal enterprise data.

As more companies adopt Cloud technologies, integrating and exposing data across Clouds, as well as across on-premise systems, will become a necessity for the businesses just to operate. Companies that succeed in defining, integrating and simplifying access to their internal enterprise data will have a competitive advantage over their rivals. The competitive advantage comes in two flavors: 1) access to raw structure data for business process needs, such as mobile and business applications and automation, and 2) big data use for reporting and predictive analytics.

There was one slide that I presented that I thought would make a good blog post that described the long-term enterprise data access strategy that companies can use as a reference blueprint. There is one major point that I like to stress for this slide: Data-as-a-Service (DaaS). To really empower the business, facilitate company growth and promote innovation, enterprise data needs to become more of a managed service. In other words, IT needs to consider a data strategy that centers around data. This includes: data governance, data definitions, data cleansing, data enrichment, data aggregation, and ultimately the data transport. While all these areas need to work in concert, it is my opinion that the data transport strategy is an important aspect that IT departments need to focus on getting started sooner rather than later. Every business is different, however implementing the various transport layers takes time and requires specific talent to achieve this objective.

Data Delivery Platform: Empowering & Scaling the Business for Adaptability

Here is the slide that contains the diagram (click the image for a larger view). Now, let’s discuss this in more detail.

Budget & Priorities

At most companies, departments within the business prioritize dollars for various initiatives. For example, the marketing department may want to implement a new direct email marketing platform, finance department may allocate dollars to revamp its reporting engine for better financial visibility or the IT department may have an initiative to automate 25% of its manual processes to help reduce expenses for the company. Whatever the business use case is, it’s clear that all the initiatives require data to be successful. It might be prospect data for a direct marketing campaign, streaming financial data for real-time analysis or simply real-time employee data for system access and organizational charts.

While many departments continue to partner with their IT counterpart, others are opting to forgo traditional IT engagements. This may be attributed to business users becoming more technology savvy or Cloud applications becoming easier to implement. Whatever the reason may be, access to enterprise data for these systems is still a requirement for these applications to be operational.

Data Delivery Platform

The key to faster data integrations and increased productivity is the Data Delivery Platform. At its core it is a set of Web Services (preferably RESTful Web Services) that enable access to raw data. This data could be custom enterprise APIs that aggregate employee, customer or financial data. It may also include streaming analytics data, internal product APIs or external partner APIs. This platform is a core service that the IT department maintains and is responsible for ensuring its availability to the organization.

You may have noticed the certified stamp in the diagram. There is tremendous value in business agility with this this type of data delivery platform, but if the data is not accurate or is incomplete it quickly becomes unusable which will contribute to low adoption. That’s why it’s key that the data be accurate and trusted. IT will need to work cross-functionally to help govern data to ensure its accuracy thereby certifying the data and increasing adoption of the Data Delivery Platform.

Self-Service Zone

The IT landscape continues to change. With business users looking to become more self-sufficient, IT needs to consider supporting more self-service services that empower its business partners. This will require IT to think differently and consider what its core services are. My diagram focuses on the data integration needs for business partners. IT will need to consider adding Platform-as-a-Service (PaaS) tools that enable the business, as well as other IT application development teams, to access data. That might be via a programming language for organizations that have technical competencies or traditional ETL tools for data migration, loading or synchronization activities. Business partners have the option to engage with consulting firms for their implementations or to continue to partner with their IT teams to leverage the same set of tools. These options help empower the business partner thereby increasing overall satisfaction, while increasing delivery (and hopefully innovation) for the company.

Final Thoughts

Business agility is the largest benefit that jumps out with a Data Delivery Platform, however there are a number of other benefits that the company can realize. With this type of architecture, companies can now separate data logic from application business logic which will help with the cost of application development and maintenance. It also decouples the source system from business applications consuming the data. This not only helps with maintenance costs, but will not disrupt applications built on top of the Data Delivery Platform provided the Web Service contract does not change.

Business moves faster today. Customers’, partners’ and employees’ expectations have changed. Your IT organization needs to adapt to the change and invest in services that enable the business to get to its data. The company that succeeds in this will not only have a major competitive advantage, but will be positioned to be a next generation company.