This version brought in some big changes. Up until now we imitated the way
Cloudera setups Kafka. As we provide more roles (Schema Registry, Kafka Connect,
Kafka REST) and support all authentication scenarios, this way of setting up
Kafka was more complex and counterintuitive to people who already know Kafka.

For this version we aligned the setup layout with Kafka, letting the
administrator to configure broker listeners. This permits more complex setups
(i.e set up any combination of listeners) with more ease and better
understanding, encouraging knowledge transfer from most Kafka resources. From
our point, it makes the setup scripts more robust and will accelerate future
development.

A small drawback is that a best practice is now almost enforced: all instances
of a role should have the same settings for each listener (basically use the
same port) unless the safety valve is used. The mirrormaker role was
removed. The kafka-mirror-maker application is still available and may be used.
We plan to re-introduce the role in a future release.

List of visible chances:

Support for Kafka 0.10.2.x, Confluent 3.2.x

Create listener configuration options for all types of listeners of all roles
and make them available service wide

All options of Kafka are now availabe in Cloudera Manager

Support all authentication scenarios for all roles and permit overrides
by instance

Fix: Kafka REST and Connect Distributed do not support auth to Schema Registry
via client certificates, so now we try to detect schema registries with http
listeners or non-auth ssl listeners and use them.

Instead of appending a unique id to each log file, now we append a
service-wide, user-defined id. This provides a common log path for all
components in all nodes, while still permitting multiple Confluent service
instances to work alongside in the cluster.

Simplify kerberized setups. If you force a plaintext port that is accessible
from all hosts in you cluster, all the services will use it automatically
without need to manually adjust bootstrap servers. Also now you can set the
brokers to communicate through the plaintext port if you set a different
security model for your clients.

Connect Classpath option lets you add custom connectors to Kafka Connect.
You only have to add them to the same path for every host that runs Connect.

Datamountaineer’s Stream Reactor
is available as a parcel. Should you choose to activate it, restart Connect
Distributed and it will find automatically the new connectors.