The Apache Ignite Integrations Developer Hub

Welcome to the Apache Ignite Integrations developer hub. You'll find comprehensive guides and documentation to help you start working with Apache Ignite Integrations as quickly as possible, as well as support if you get stuck. Let's jump right in!

Streaming Data via Kafka Connect

IgniteSinkConnector will help you export data from Kafka to Ignite cache by polling data from Kafka topics and writing it to your specified cache.Connector can be found in 'optional/ignite-kafka.' It and its dependencies have to be on the classpath of a Kafka running instance, as described in the following subsection.For more information on Kafka Connect, see Kafka Documentation.

where cacheName is the name of the cache you specify in '/some-path/ignite.xml' and the data from 'someTopic1,someTopic2' will be pulled and stored. cacheAllowOverwrite can be set to true if you want to enable overwriting existing values in the cache. If you need to parse the incoming data and decide on your new key and value, you can implement it as StreamSingleTupleExtractor and specify as singleTupleExtractorCls, as shown above.You can also set cachePerNodeDataSize and cachePerNodeParOps to adjust per-node buffer and the maximum number of parallel stream operations for a single node.

See example-ignite.xml in tests for a simple cache configuration file example.

Streaming data with Ignite Kafka Streamer Module

If you are using Maven to manage dependencies of your project, first of all you will have to add Kafka Streamer module dependency like this (replace '${ignite.version}' with actual Ignite version you are interested in):