Microservices and Your Data: What You Need To Know

In this special guest feature, Christian Posta, Chief Architect, Cloud Application Development at Red Hat, argues that data is one of the hardest parts of microservices. If a core tenet of microservices is that each service should have its own database, how does the system reconcile changes across these databases while maintaining independence and autonomy? The answer, according to Christian, is to adopt a domain-driven design approach. By modelling the data within a set domain and context, microservices can shed these dependencies while still maintaining data consistency throughout the system. Christian has spent time at web-scale internet companies, as a specialist consultant in the integration and messaging space, as well as at traditional enterprises in the manufacturing, power-distribution, retail, and financial-services space. Christian now works closely with Red Hat’s strategic customers and partners to realize business value through technology by implementing cloud-native solutions using open source including Red Hat’s extensive open-source portfolio. Christian guides customers covering topics such as high-performance messaging, integration, caching, stream processing, data-intensive architectures and how we deliver those in a cloud-native way with containers.

Microservices has been a popular and “buzzwordy” topic for the last couple of years across the blogosphere, conference circuit, and social media. Microservices can help improve technology architecture as well as team organization that fits well with DevOps and enables an organization to scale its software development and move faster. However, one of the hardest parts of systems architecture is how to deal with data. Taking a closer look at a system’s data and how services may interact with each other in order to share that data can help smooth out some of the rough parts an organization may face as part of its microservices journey.

Defining the domain

Martin Fowler and the Thoughtworks folks helped coin the term “microservices” beginning in 2011 after they observed some teams which implemented some of these practices in the same common architectural style as what they had been exploring.These observations have gained momentum since being publically presented in 2012, after which Netflix dubbed the concepts as “fine grained SOA.” In the right context, these concepts may be fine, but before an organization starts making architecture decisions, it should understand the problem it is trying to solve.

To establish the context and gain a proper understanding of the problem that needs to be solved, the organization needs to understand that microservices will likely introduce massive complexity into both its business and technical domains, and neither can be ignored over the other. Traditional enterprises are likely to face both infrastructure and technology scale that may have many contradicting and ambiguous areas. This is where the patterns and practices from Domain Driven Design (DDD) can help. DDD can help unpack the business problems and draw boundaries around them. From there, the organization can start to architect its services and decide how the domain models should share their data.

Making time boundaries explicit

Once a strategy is in place for how the system boundaries should be architected, the team should give some thought to how the services will communicate between the boundaries and how they should treat data changes and updates. Even in a system where there are boundaries around the domain and data, the idea of “shared data” presents additional challenges. For example, a customer may be modeled a few different ways depending on the domain boundaries (e.g., the customer will be seen differently in a loyalty program than in an Accounts Receivable domain), but there may be some concepts that can be related. An interesting property of the autonomy and boundaries given to these parts of the system is that they each live in its own idea of what is “now” and anything that happens outside of these boundaries happened “then” or in the past. Organizations can explore using event-driven design concepts to communicate between these boundaries by passing events, and make “time” a first-class part of the design.

Why microservices?

It is important to regularly pause and ask, “Why are we doing this?” Microservices can help break a system down into parallelizable streams of work and enable development teams to iterate faster. However, understanding the organization, domain, and technical complexities that need to be solved are all parts of the equation. There are patterns and practices, like those from the DDD community, that help ground microservices projects in reality and address some of these problems. There are other architecture patterns, such as using events to communicate between services, that can also serve to alleviate some of the technical challenges of purely synchronous interactions. Focusing on the domain and the data is a good place to start when going down the microservices path.

Resource Links:

Industry Perspectives

In this special guest feature, Sean McDermott, CEO and founder of Windward Consulting Group and RedMonocle, offers what enterprises need to know about the five levels of AIOps maturity. When maneuvering through each level, keep the long-term AIOps strategy and goals at the center to achieve the true potential of AIOps.

Latest Video

White Papers

This whitepaper from Imply Data Inc. introduces Apache Druid and explains why delivering real-time analytics on a data lake is so hard, approaches companies have taken to accelerate their data lakes, and how they leveraged the same technology to create end-to-end real-time analytics architectures.