Topics

Featured in Development

Understandability is the concept that a system should be presented so that an engineer can easily comprehend it. The more understandable a system is, the easier it will be for engineers to change it in a predictable and safe manner. A system is understandable if it meets the following criteria: complete, concise, clear, and organized.

Featured in Architecture & Design

Sonali Sharma and Shriya Arora describe how Netflix solved a complex join of two high-volume event streams using Flink. They also talk about managing out of order events and processing late arriving data, exploring keyed state for maintaining large state, fault tolerance of a stateful application, strategies for failure recovery, data validation batch vs streaming, and more.

Featured in Culture & Methods

Tim Cochran presents research gathered from ThoughtWorks' varied clients and projects, and shows some of the metrics their teams have identified as guides to creating the platform and the culture for high performing teams.

Information is a live stream of datasets in raw format
Streaming data is a useful technique when you have to quickly react to changes in your business, and for Vernon, it's currently the best way to deal with data in real-time. He notes though that claiming that data is always streaming in the cloud is not accurate. Most business systems still use persistent state – data at rest.

When working with streams, he also notes that there are smart solutions that can help in parsing streaming data and warns against technology stacks that lacks these kinds of tooling.

The skillset at traditional Java shops is not conducive to modern, cloud-based applications
Vernon emphasizes that the skills of experienced developers can be leveraged in a reactive architecture. There is no need for companies to replace their current developers and architects with new ones just to embrace a new technology stack, and putting core business at risk in doing so.

Vernon notes though that in order for the developers to improve, they should be provided with modern tooling that for instance supports concurrent and parallel hardware, a commodity nowadays.

Data is not persisted; it's always on the fly
Vernon claims that this not true; data is persisted similar to how it has been done for decades. If there is an outage and data is not persisted, the most valuable asset of a company will be lost. This is often also true for streams, and for the same reason. There are of course exceptions to this; sometimes there is only a need for samples of data in a stream, in which case parts of the stream can be lost without any impact.

You should not consider it inferior to use an RDBMS when circumstances call for one. In some cases, an RDBMS may not be able to meet the scalability needs of a specific set of use cases. At other times it’s just perfect for the task at hand, and if arbitrarily ignored as a technology choice could lead to much unnecessary complexity.

For Vernon, it’s important to understand when persistence is needed, and notes that most business experts and developers can reason about this and make sound decisions.

Data is a stream, you can’t run a SQL query on it
This is true for the stream itself, but commonly a stream of data is projected into a store that enables querying using SQL. One example of querying live streaming data is known as continuous query and Vernon notes that this has been available for quite a while.

Reactive is the new way of building applications, monoliths get no advantage of the cloud
Together with a claim that a monolithic application gets no benefits of the cloud, these are for Vernon common but erroneous claims, and he refers to a technique called lift and shift for achieving the benefits of the cloud while preserving a legacy system. For Vernon, a properly modularized monolith is far better than a large array of strongly coupled microservices. Instead, problems arise from weak architectures and poor modularity which often leads to a big ball of mud.

Vernon recommends looking into microservices when the rate of change across business domains differs substantially, and notes the importance of understanding when microservices or a monolith is the preferred architecture of an application.

When using Java, you must do things differently and in a reactive way to leverage advantages of the cloud
Vernon believes that Java or another language on the JVM is very suitable for creating reactive applications as well as for implementing streaming and real-time, data driven use cases. He also believes that Java has a bright future and refers to a quote by Bjarne Stroustrup:

There are only two kinds of [programming] languages: the ones people complain about and the ones nobody uses.

Related Editorial

Related Sponsored Content

Related Sponsor

Aerospike is the global leader in next-generation, real-time NoSQL data solutions for any scale. Aerospike’s patented Hybrid Memory Architecture™ delivers an unbreakable competitive advantage by unlocking the full potential of modern hardware, delivering previously unimaginable value from vast amounts of data at the edge, to the core and in the cloud. Learn more.