Re: Data Integration Talend studio ElasticSearch ?

Hi,

We have tElasticSearchInput and tElasticSearchOutput component in Big Data Spark Batch and Spark Streaming. You need to use a paid subscription Big Data or Realtime Big Data edition to get these components. We do not have ElasticSearch in traditional DI product.

Re: Data Integration Talend studio ElasticSearch ?

Of course, you can potentially write this code and achieve connectivity with a Java API to ElasticSearch and Code Routines in Talend Studio. Your Code Routine will wrap the ElasticSearch API.

However, how complex will it be and how much time/effort it will take to write? I don't know. And remember about Avro Schemas etc that you may need to handle.

If you need to process 5 -10 million rows per hour, then your best approach is to do this in Spark Batch/Streaming. This way you can leverage the capability of a hadoop/spark cluster which can enable to meet this requirements.

Re: Data Integration Talend studio ElasticSearch ?

Hi,

We have tElasticSearchInput and tElasticSearchOutput component in Big Data Spark Batch and Spark Streaming. You need to use a paid subscription Big Data or Realtime Big Data edition to get these components. We do not have ElasticSearch in traditional DI product.

Re: Data Integration Talend studio ElasticSearch ?

Of course, you can potentially write this code and achieve connectivity with a Java API to ElasticSearch and Code Routines in Talend Studio. Your Code Routine will wrap the ElasticSearch API.

However, how complex will it be and how much time/effort it will take to write? I don't know. And remember about Avro Schemas etc that you may need to handle.

If you need to process 5 -10 million rows per hour, then your best approach is to do this in Spark Batch/Streaming. This way you can leverage the capability of a hadoop/spark cluster which can enable to meet this requirements.