Apache Airflow is an open-source Python tool for orchestrating data
processing pipelines. In each workflow tasks are arranged into a
directed acyclic graph (DAG). Shape of this graph decides the overall
logic of the workflow. A DAG can have many branches and you can
decide which of them to follow and which to skip at execution time.

This creates a resilient design because each task can be retried
multiple times if an error occurs. Airflow can even be stopped
entirely and running workflows will resume by restarting the last
unfinished task. Logs for each task are stored separately and are
easily accessible through a friendly web UI.

In my talk I will go over basic Airflow concepts and through examples
demonstrate how easy it is to define your own workflows in Python
code. We'll also go over ways to extend Airflow by adding custom task
operators, sensors and plugins.