Architecting large volume streaming data solution in python.

Aniket Dalal (~aniket2)
|
30 May, 2015

2

Votes

Description:

Solutions developed for web, especially over social network have started seeing demand for large scale real time streaming data analytics. This requires building applications on distributed platform that can scale, reliably process data without any loss, satisfy functional needs and at the same time meet the strict latency requirements .

This talk aims at discussing use case for large volume streaming data analysis. Goal of this session would be to cover:

Challenges of developing streaming data architecture.

Defining streaming data use case and distributed architecture for it.

Brief introduction to Kafka and its application in solving streaming data retention.

Brief Introduction to Spark and its application in distributed computing.

0

0

The objective is to expose audience on how to build big data streaming architecture. The talk would include 3 aspects: Challenges of building streaming architecture, tools and technologies for building such an architecture and finally Python libraries and tools available that can be used.

Focus is on big data design paradigm and tools available in python to facilitate it.