Version compatibility matrix

scalatest-embedded-kafka is available on Bintray and Maven Central, compiled for both Scala 2.11 and 2.12. Scala 2.10 is supported until 0.10.0. Scala 2.12 is supported from 0.11.0 onwards, following Apache Kafka release cycle.

Currently there's no support for Scala 2.13-Mx as Kafka artifacts are not published for these versions.

Starting from 1.0.0, versions match the version of Kafka they're built against. However in the past there were some mismatches - the easiest way is to check through the git history of the build.sbt file

If you're using Kafka 1.1.0, please use version 1.1.0-kafka1.1-nosr - this version doesn't pull in the Confluent Schema Registry by default

In-memory Zookeeper and Kafka will be instantiated respectively on port 6000 and 6001 and automatically shutdown at the end of the test.

Use without the withRunningKafka method

A EmbeddedKafka companion object is provided for usage without extending the EmbeddedKafka trait. Zookeeper and Kafka can be started and stopped in a programmatic way. This is the recommended usage if you have more than one test in your file and you don't want to start and stop Kafka and Zookeeper on every test.

If you want to run ZooKeeper and Kafka on arbitrary available ports, you can use the withRunningKafkaOnFoundPort method. This is useful to make tests more reliable, especially when running tests in parallel or on machines where other tests or services may be running with port numbers you can't control.

This works for withRunningKafka, withRunningKafkaOnFoundPort, and EmbeddedKafka.start()

Also, it is now possible to provide custom properties to the broker while starting Kafka. EmbeddedKafkaConfig has a customBrokerProperties field which can be used to provide extra properties contained in a Map[String, String]. Those properties will be added to the broker configuration, be careful some properties are set by the library itself and in case of conflict the customBrokerProperties values will take precedence. Please look at the source code to see what these properties are.

Utility methods

The EmbeddedKafka trait provides also some utility methods to interact with the embedded kafka, in order to set preconditions or verifications in your specs:

How to use

For most of the cases have your Spec extend the EmbeddedKafkaStreamsAllInOne trait. This offers both streams management and easy creation of consumers for asserting resulting messages in output/sink topics.

If you only want to use the streams management without the test consumers just have the Spec extend the EmbeddedKafkaStreams trait.

Use the runStreamsWithStringConsumer to:

Create any topics that need to exist for the streams to operate (usually sources and sinks).

Pass the Topology that will be used to instantiate and start the Kafka Streams. This will be done while using the withRunningKafka closure internally so that your stream runs with an embedded Kafka and Zookeeper.

Pass the {code block} that needs a running instance of your streams. This is where your actual test code will sit. You can publish messages to your source topics and consume messages from your sink topics that the Kafka Streams should have generated. This method also offers a pre-instantiated consumer that can read String keys and values.

For more flexibility, use runStreams and withConsumer. This allows you to create your own consumers of custom types as seen in the example test.

Utility methods

Using streams

For most of the cases have your Spec extend the EmbeddedKafkaStreamsWithSchemaRegistryAllInOne trait. This offers both streams management and easy creation of consumers for asserting resulting messages in output/sink topics.

If you only want to use the streams management without the test consumers just have the Spec extend the EmbeddedKafkaStreamsWithSchemaRegistry trait.