1. Cloud architects
2. Systems engineers
3. Developers working with GCE

1. Knowledge of SQL
2. Experience with data processing tools
3. Basic understanding of cloud services

Course Outline:

Module 1: Introduction to Data Engineering on GCP

1. Role of a data engineer
2. Data engineering challenges
3. Data lakes vs data warehouses
4. GCP data ecosystem overview
5. Case study: Real-world GCP customer pipeline

Module 2: Data Ingestion

1. Introduction to Cloud Pub/Sub
2. Batch vs streaming ingestion patterns
3. Basics of Cloud Dataflow pipelines
4. Lab: Publishing and processing streaming data with Pub/Sub & Dataflow

Module 3: Data Storage

1. Cloud Storage (buckets, lifecycle, security)
2. Relational storage: Cloud SQL, Spanner
3. NoSQL storage: Datastore/Firestore, Bigtable
4. Lab: Loading structured/unstructured data into GCP storage services

Module 4: Data Warehousing with BigQuery

1. Introduction to BigQuery as a modern data warehouse
2. Loading and querying data
3. Partitioning, clustering, schema design
4. Lab: Running federated queries on external datasets

Module 5: Batch Data Processing

1. ETL/ELT concepts on GCP
2. Quality considerations and transformations
3. Using Dataproc for Spark and Hadoop workloads
4. Lab: Running Spark jobs on Dataproc

Module 6: Streaming Data Processing

1. Streaming concepts and challenges
2. Dataflow streaming pipelines
3. BigQuery streaming inserts
4. Cloud Bigtable for high-throughput streaming
5. Lab: Real-time analytics dashboard with Pub/Sub + Dataflow + BigQuery

Module 7: Data Orchestration and Automation

1. Cloud Data Fusion for visual ETL design
2. Cloud Composer (Apache Airflow) for orchestration
3. Scheduling workflows and monitoring pipelines
4. Infrastructure automation with Deployment Manager and Terraform (intro)
5. Lab: Building a pipeline with Data Fusion and Composer

Module 8: Monitoring, Security, and Governance

1. Cloud Monitoring, Logging, Error Reporting, Tracing
2. Data security and access control with IAM and DLP API
3. Governance best practices (projects, quotas, billing)
4. Lab: Detecting PII with Cloud DLP

Module 9: Analytics and Machine Learning with BigQuery

1. BigQuery ML: SQL-based ML models
2. Common supported models
3. Performance tuning for large queries
4. Lab: Building a predictive model with BigQuery ML

Module 10: AI/ML Services for Data Enrichment

1. Pre-built ML APIs (Vision, NLP, Translation) for unstructured data
2. Cloud AI Platform Notebooks for data exploration
3. AutoML for custom ML models
4. Kubeflow pipelines for production ML workflows
5. Lab: Classifying text data with Cloud NLP API

Module 11: Containers and Modern App Integration

1. Using GCP with containerized workloads
2. Google Kubernetes Engine (GKE) for scalable data apps
3. Cloud Run for serverless data services
4. Cloud Pub/Sub + Functions for event-driven data processing

Training material provided: Yes (Digital format)