Interface IgniteDataStreamer<K,V>

Data streamer is responsible for streaming external data into cache. It achieves it by
properly buffering updates and properly mapping keys to nodes responsible for the data
to make sure that there is the least amount of data movement possible and optimal
network and memory utilization.

Note that streamer will stream data concurrently by multiple internal threads, so the
data may get to remote nodes in different order from which it was added to
the streamer.

Also note that IgniteDataStreamer is not the only way to add data into cache.
Alternatively you can use IgniteCache.loadCache(IgniteBiPredicate, Object...)
method to add data from underlying data store. You can also use standard
cache put(...) and putAll(...) operations as well, but they most
likely will not perform as well as this class for adding data. And finally,
data can be added from underlying data store on demand, whenever it is accessed -
for this no explicit data adding step is needed.

IgniteDataStreamer supports the following configuration properties:

perNodeBufferSize(int) - when entries are added to data streamer via
addData(Object, Object) method, they are not sent to in-memory data grid right
away and are buffered internally for better performance and network utilization.
This setting controls the size of internal per-node buffer before buffered data
is sent to remote node. Default is defined by DFLT_PER_NODE_BUFFER_SIZE
value.

perNodeParallelOperations(int) - sometimes data may be added
to the data streamer via addData(Object, Object) method faster than it can
be put in cache. In this case, new buffered stream messages are sent to remote nodes
before responses from previous ones are received. This could cause unlimited heap
memory utilization growth on local and remote nodes. To control memory utilization,
this setting limits maximum allowed number of parallel buffered stream messages that
are being processed on remote nodes. If this number is exceeded, then
addData(Object, Object) method will block to control memory utilization.
Default is defined by DFLT_MAX_PARALLEL_OPS value.

autoFlushFrequency(long) - automatic flush frequency in milliseconds. Essentially,
this is the time after which the streamer will make an attempt to submit all data
added so far to remote nodes. Note that there is no guarantee that data will be
delivered after this concrete attempt (e.g., it can fail when topology is
changing), but it won't be lost anyway. Disabled by default (default value is 0).

allowOverwrite(boolean) - Sets flag enabling overwriting existing values in cache.
Data streamer will perform better if this flag is disabled, which is the default setting.

receiver(StreamReceiver) - defines how cache will be updated with added entries.
It allows to provide user-defined custom logic to update the cache in the most effective and flexible way.

deployClass(Class) - optional deploy class for peer deployment. All classes
streamed by a data streamer must be class-loadable from the same class-loader.
Ignite will make the best effort to detect the most suitable class-loader
for data loading. However, in complex cases, where compound or deeply nested
class-loaders are used, it is best to specify a deploy class which can be any
class loaded by the class-loader for given data.

parallelOps - Maximum number of parallel stream operations for a single node.

autoFlushFrequency

long autoFlushFrequency()

Gets automatic flush frequency. Essentially, this is the time after which the
streamer will make an attempt to submit all data added so far to remote nodes.
Note that there is no guarantee that data will be delivered after this concrete
attempt (e.g., it can fail when topology is changing), but it won't be lost anyway.

autoFlushFrequency

void autoFlushFrequency(long autoFlushFreq)

Sets automatic flush frequency. Essentially, this is the time after which the
streamer will make an attempt to submit all data added so far to remote nodes.
Note that there is no guarantee that data will be delivered after this concrete
attempt (e.g., it can fail when topology is changing), but it won't be lost anyway.

future

Gets future for this streaming process. This future completes whenever method
close(boolean) completes. By attaching listeners to this future
it is possible to get asynchronous notifications for completion of this
streaming process.

Returns:

Future for this streaming process.

deployClass

Optional deploy class for peer deployment. All classes added by a data streamer
must be class-loadable from the same class-loader. Ignite will make the best
effort to detect the most suitable class-loader for data loading. However,
in complex cases, where compound or deeply nested class-loaders are used,
it is best to specify a deploy class which can be any class loaded by
the class-loader for given data.