Producers are data-generating applications, such as sensors in automobiles or activity loggers in servers. Producers create
messages with the collected data and publishes the messages to MapR Event Store For Apache Kafka topics, specifically, to MapR Event Store For Apache Kafka topic-partitions.

The option to automatically set up stream replication, also known as replica autosetup, performs the steps to set up and
start the replication of streams. The replica autosetup option is available through the MCS and CLI.

The replication state indicates when stream replication is in progress and it also displays the status of operations related
to replica autosetup with directcopy. The maprcli stream replica list command displays the following replication states.

When replicating streams, MapR Event Store For Apache Kafka replicates messages that are published to a source stream. Gateways are services that receive messages from source streams
and publish them in replica streams.

Stream Replication

You can replicate streams to other MapR clusters worldwide or to other streams within a
MapR cluster.

There are many different scenarios in which replicating MapR Event Store For Apache Kafka streams can be
useful.

Basic Master-Slave Replication

For example, suppose that your company has a factory in Nagoya, and sensors in the equipment
track different metrics. The sensors are producers publishing messages to a stream named
metrics. The applications that use the collected metrics would read the
messages from the stream, playing the role of consumers. With replication, the factory could
create a stream in the nagoya cluster and maintain a backup of the stream in
the nagoya_ha cluster.

This type of replication is called basic master-slave replication because
replication is in one direction only. The metrics stream in the nagoya_ha
cluster is considered to be a replica. The original metrics stream is considered
to be the upstream source for the replica. This type of replication is simple to
set up with the command maprcli stream replica autosetup.

Suppose further that your company also has a factory in Kaesong which also collects metrics
from its equipment, perform analyses on the data, and replicates its own
metrics streams to a backup.

Many-to-One Replication

Your company's headquarters are in San Francisco and you want data analysts there to be able
to perform analyses of all data company-wide. You can replicate the two
metrics streams that are in the your factories to the
metrics stream in the sanfrancisco cluster. In this
scenario, the replica is the metrics stream in the
sanfrancisco cluster. This replica has two upstream sources: the
metrics streams that are replicated from the two factories.

This type of replication, called many-to-one replication, requires that the
topics in each stream have unique names, so that message offsets do not conflict. For example,
suppose both factories have an assembly line named Line 2 and the topic in each factory's
stream for collecting metrics from this line is named line_2. At some point, the Nagoya
factory and the Kaesong factory both replicate messages that use the same offsets. Because
offsets are replicated together with messages, messages can be overwritten in this case.

To avoid this type of problem, the sensors for Line 2 in the Nagoya factory might publish to
a topic named line_2_nagoya, the sensors for Line 2 in the Kaesong factory
might publish to a topic named line_2_kaesong, and so on. The consolidated
stream in San Francisco would contain the topics line_2_nagoya and
line_2_kaesong.

Multi-Master Replication

Another type of of replication that can be useful is multi-master replication.
You can use it when you need two streams both to send updates to and receive updates from the
other stream. Each stream is a replica and an upstream source. MapR Event Store For Apache Kafka keeps both
streams synchronized with each other. This type of replication is also simple to set up with
the command maprcli stream replica autosetup.

As with many-to-one replication, the names of the topics in each stream must be unique across
both streams, so that offsets for messages do not conflict.

Replica Autosetup for Streams
The option to automatically set up stream replication, also known as replica autosetup, performs the steps to set up and start the replication of streams. The replica autosetup option is available through the MCS and CLI.

States of Stream Replication
The replication state indicates when stream replication is in progress and it also displays the status of operations related to replica autosetup with directcopy. The maprcli stream replica list command displays the following replication states.

Gateways and Stream Replication
When replicating streams, MapR Event Store For Apache Kafka replicates messages that are published to a source stream. Gateways are services that receive messages from source streams and publish them in replica streams.