If you are building reactive microservices, you would probably have to merge data streams from different source APIs into a single result stream. It inspired me to create this article containing some of the most common scenarios of using reactive streams in microservice-based architecture during inter-service communication. I have already described some aspects related to reactive programming with Spring based on Spring WebFlux and Spring Data JDBC projects in the following articles:

Spring Framework supports reactive programming since version 5. That support is build on top of Project Reactor. Reactor is a fourth-generation Reactive library for building non-blocking applications on the JVM based on the Reactive Streams Specification. Working with this library can be difficult at first, especially if you don’t have any experience with reactive streams. Reactive Core gives us two data types that enable us to produce a stream of data: Mono and Flux. With Flux,we can emit 0..nelements. While with Mono, we can create a stream of 0..1 elements. Both those types implement the Publisher interface. Both of these types are lazy, which means they won’t be executed until you consume it. Therefore, when building reactive APIs, it is important not to block the stream. Spring WebFlux doesn’t allow that.

Introduction

The sample project is available on GitHub in repository reactive-playground. It is written in Kotlin. In addition to some Kotlin libraries, the only single dependency that needs to be added in order to use Project Reactor is reactor-core.

I would not like to show you the features of Project Reactor based on simple String objects, like in many other articles. Therefore, I have created the following class hierarchy for our tests that allows us to simulate APIs built for three different domain objects.

Class Organization contains a list of Employee and Department. Each department contains a list of Employee assigned only to the given department inside organization. Class Employee has properties: organizationId that assigns it to the organization and departmentId that assigns it to the department.

We would like to return the single stream emitting organization that contains a list of employees, as shown below.

Here’s the solution. We use the zipWhen method that waits for result from source Mono and then calls the second Mono. Because we can zip only the same stream types (in that case these are Mono), we need to convert Flux<Employee> returned by the getEmployeesByOrganization method into Mono<MutableList<Employee>> using collectList function. Thanks to zipWhen, we can then combine two Mono streams and create a new object inside the map function.

The goal is to merge those two streams and return the single Flux stream, emitting departments that contain all employees assigned to the given department. Here’s the picture that illustrates the transformation described above.

We can do that in two ways. First, we call the flatMap function on stream with departments. Inside flatMap, we zip every single Department with stream of employees. That stream is then filtered by the departmentId and converted into the Mono type. Finally, we create the Mono type using the map function that emits the department containing the list of employees.

The second way groups Flux with employees by departmentId. Then, it invokes zipping and mapping functions similar to the previous approach.

Scenario 3

This scenario is simpler than the two previous scenarios. We have two API methods that emit Flux with the same object types. The first of them contains a list of employees — id, name, salary properties— while the second includes the id, organizationId, and departmentId properties.

We would like to merge those two streams into a single stream ordered by id. The following picture shows that transformation.

Here’s the solution. We use the mergeOrderedWith function with comparator that compares id. Then, we can perform some transformations on every object, but it is only an option that shows the usage on map function.

Scenario 5

And, the last scenario in this article – we have a single input stream Mono<Organization> that contains a list of departments. Each department inside that list contains the list of all employees assigned to the given department. Here’s our API method implementation.

The goal is to convert the stream to the same stream Flux<Department>, but containing the list of all employees in department. The following picture visualizes the described transformation.

Here’s the solution. We invoke the flatMapIterable function that converts Flux<Department> into Flux<Employees> by returning List<Employee>. Then, we convert it to Mono and add to newly created Organization object inside the map function.