The hostname and network options ensure that you can easily address Apache Kafka from the pipeline later on.

Step 2: Add and configure Kafka Producer

Open the pipeline which you have created during the previous tutorials (test.myFirstPipeline) in the modelling environment http://localhost:8090.

Remove the connection between the Data Generator operator and the Terminal operator.

Add a Kafka Producer operator to the pipeline by drag & drop. Then connect the output port of the Data Generator operator to the message port of the Kafka Producer operator.

Right click on the Kafka Producer and go to “Open Configuration” to configure the operator properties . You need to maintain the following properties:

Property

Value

brokers

kafka:9092

topic

sensordata

Step 3: Add and configure Kafka Consumer

Add a Kafka Consumer2 operator to the pipeline by drag & drop.Add a ToString Converter operator to the pipeline by drag & drop.Connect the message port of the Kafka Consumer2 operator to the ininterface port of the ToString Converter operator.Connect the outstring port of the ToString Converter operator to the in1 port of the Terminal operator.

Right click on the Kafka Consumer2 and go to “Open Configuration” to configure the operator properties . You need to maintain the following properties:

Property

Value

brokers

kafka:9092

topics

sensordata

Afterwards press the Save button.

Step 4: Execute the data pipeline

Press the Run button to execute the pipeline.

When the Status tab indicates that the pipeline is running, use the context menu Open UI of the Terminal operator to see the generated sensor data.

In contrast to the previous tutorial, this time the generated sensor data is not sent from the Data Generator operator to the Terminal operator directly, but via Apache Kafka.