• People being curious about data engineering.
• People who want to learn basic and advanced concepts about Apache Airflow.
• People who like hands-on approach.

• Python Programming
• Basic Understanding of Workflow Management

Course Outline:

Introducing Apache Airflow                                                       

• What Airflow is and what does it solve?
• Airflow architecture
• How do we represent a Pipeline?
• Demo: Our first DAG
• Tasks, TaskFlow and Operators
• Demo: First Pipeline
• Capstone Lab

Mastering scheduling

• execution_date, start_date and schedule_interval
• Handling non-default schedule_intervals
• Demo: Playing with time
• Capstone Lab

Abstracting functionality

• Using custom operators
• Creating TaskGroups vs subDAGs
• Sharing data with xCOMs
• Branching and Triggers
• Sensors and SmartSensors
• Capstone Lab

Executors and Scaling Airflow

• Abandoning SQLite to PostgreSQL
• Executors: Debug, Local, Celery
• Concurrency and parallelism
• Demo: Concurrency with Celery
• Airflow in Kubernetes, the old and new ways
• KEDA and HA scheduler
• Demo: Deploying a highly availability fault-tolerant Airflow

Hackathon

Training material provided: Yes (Digital format)