This course is ideal for:

1. Data Engineers and Streaming Pipeline Developers
2. Google Cloud Developers and Architects
3. Engineers migrating from Spark, Flink, or Airflow
4. Developers building real-time analytics and ETL workflows

1. Basic programming knowledge (Python or Java)
2. Familiarity with cloud concepts (preferably Google Cloud)
3. Some experience with data processing frameworks (e.g., Spark, Hadoop) is helpful 

Course Outline:

Module 1: Introduction to Apache Beam

• Evolution of Big Data Frameworks
• Overview and use cases of Apache Beam
• Apache Beam Architecture and SDKs
• Beam’s portable and unified programming model

Module 2: Beam Setup and Basic Concepts

• Key abstractions: PCollection, PTransform, Pipeline
• Installing Beam and setting up dev environment
• Building your first Beam pipeline (local runner)

Module 3: Working with Beam Transforms

• Structure of a Beam pipeline
• Input transforms: Read, Create
• Output transforms: Write to files, databases
• Core transforms: Map, FlatMap, Filter
• Hands-on: Basic read-transform-write pipeline

Module 4: Pipeline Logic and Advanced Transforms

• ParDo, DoFn and branching logic
• Composite transforms: abstraction and reuse
• Aggregations: Combine, CombinePerKey
• Joins with CoGroupByKey
• Hands-on: Branching + CoGroupByKey joins

Module 5: Side Inputs and Outputs

• Working with side inputs for auxiliary data
• Creating multiple outputs from one transform
• Hands-on: Multi-output transformation and filtering

Module 6: Case Study – Identifying Bank Defaulters

• Understanding credit card and loan data
• Creating modular pipelines for different defaulter types
• Hands-on: Implement pipeline to flag defaulters

Module 7: Type Hints and Coders in Beam

• What is type safety in data pipelines?
• Using Coder class for serialization
• Type hints in Beam and how Beam ensures type safety
• Hands-on: Using type hints and coders in a custom pipeline

Module 8: Introduction to Streaming in Beam

• Event-driven processing vs. batch workflows
• Pub/Sub architecture and flow
• Windowing and watermarking (intro only, optional deep dive)
• Hands-on: Pub/Sub demo with sample topic/stream

Module 9: Apache Beam with Google Cloud Dataflow

• Connecting Beam pipelines to GCP
• Creating and configuring Pub/Sub topics
• Running batch and streaming jobs on Dataflow
• Hands-on: Run a streaming pipeline from Pub/Sub to GCS or BigQuery

Training Material Provided:

• Course slides and reference guides