Any reasonably complex analysis isn’t just going be computed in a single pipeline, but instead a chain of pipelines. We often refer to chains of pipelines as dependency graph or DAG (directed acyclic graph). Before we jump into dealing with chains of pipelines, it’s important to understand how pipelines deal with multiple inputs.

A pipeline is allowed to have multiple inputs. The important thing to understand is what happens when a new commit comes into one of the input repos. In short, a pipeline processes the cross product of its inputs. We will use an example to illustrate.

Consider a pipeline that has two input repos: foo and bar. foo uses the file/incremental method and bar uses the reduce method.