Apache: Big Data Europe 2016
Click here to Register or for more information 
Back To Schedule
Wednesday, November 16 • 12:00 - 12:50
Get in Control of Your Workflows with Apache Airflow - Christian Trebing, Blue Yonder

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Whenever you work with data, sooner or later you stumble across the definition of your workflows. At what point should you process your customeräó»s data? What subsequent steps are necessary? And what went wrong with your data processing last Saturday night?

At Blue Yonder we use Apache Airflow to solve these problems. It can be extended with new functionality by developing plugins in Python. With Airflow, we define workflows as directed acyclic graphs and get a shiny UI for free. Airflow comes with some task operators which can be used out of the box to complete certain tasks. For more specific cases, you can also develop new operators in your plugin.

This talk will explain the concepts behind Airflow, demonstrating how to define your own workflows and how to extend the functionality. Youäó»ll also get to hea about our experiences using this tool in real-world scenarios.


Christian Trebing

Senior Software Engineer
Christian is a Software Developer from Karlsruhe, Germany. He has studied Computer Science at TU Darmstadt. Currently he is working on big data applications at Blue Yonder, enjoying the challenges at the intersection between software engineering and data science.

Wednesday November 16, 2016 12:00 - 12:50 CET