Pancake Stack: End-to-End, Real-time ML and AI Pipeline

InfoQ has an interesting read on the last-mile of theApache Spark machine learning pipeline, “Chris Fregly on the PANCAKE STACK Workshop and Data Pipelines”.

Its interesting to read that even Netflix and other struggled in this space:

The idea for PipelineIO dates back a few years to my time at Netflix where we were forced to build a custom ML prediction/serving layer. There wasn’t – and still isn’t, in my opinion – a production-ready, fault-tolerant, and low latency open source system to serve Netflix-scale predictions and recommendations in real-time