Introduction to Apache Airflow: author, schedule and monitor workflows

Speaker: Esperanza Buitrago
Event: Introduction to Apache Airflow: author, schedule and monitor workflows
Date: 22 February 2022
Type: Workshop
Language: English
Source: The announcement of the day on Meetup

About this talk

Apache Airflow is one of the most popular tools for implementing data engineering, machine learning, and DevOps workflows, and it is used by more than 200 companies such as Airbnb, Yahoo, Paypal, Intel or Stripe.
Airflow allows to programmatically author, schedule and monitor workflows, which are sequence of tasks, started on a schedule or triggered by an event, and it is frequently used to handle big data processing pipelines.
We are going to work with Airflow from scratch and write our first DAG, accordingly to:
airflow operators,
operators that we write by ourselves,
our own pipelines, using a small operator.
If time permits, we are also going to find out how to add operators using MongoDB operators.

Links (1)

Site