This course is designed for Data Analysts, Software Engineers, Data Engineer, Data Professional, Business Intelligence Developer, Data Architect, DevOps Engineer

• Experience of programming language like Python required.
• SQL and Data knowledge
• Familiarity with Big data is a plus

Course Outline

Day 1: –

Big Data overview

• A brief history of Big Data
• History and background of Big Data and Hadoop
• 5 V’s of Big Data
• Secret Sauce of Big Data Hadoop
• Big Data Distributions in Industry
• End-to-End Big Data Life cycle overview
• Industry Use cases

Big Data Ecosystem before Spark

• Big Data Ecosystem before Apache Spark
• Storage options – HDFS and No-SQL
• Processing options – MapReduce, Hive etc.
• Administrative tools – Zookeeper, Ozzie etc.
• Ingestion tools – Sqoop, Flume

Big Data Ecosystem after Spark

• Big Data Ecosystem after Apache Spark
• Compare MapReduce Vs Apache Spark
• Apache Spark Architecture
• Understand Apache Architecture and Libraries like Streaming, Machine Learning with Spark ML, GraphX/GraphFrames, etc.
• Understanding Spark RDD
• Setup Account on Apache Spark Databricks Cloud.
• Introduction to Notebooks concept on Databricks
• Demos and Labs

Days 2: –

Getting Started with Apache Spark

• Introduction to Spark RDD
• Spark RDD Transformation and Actions
• Spark Lifecycle
• Spark Caching
• Lab – Spark RDD Transformation & Actions
• Lab – Spark RDD Advanced Transformation & Actions
• Demos and Labs

Apache Spark SQL, DataFrames, Datasets

• Introduction to Spark SQL
• SQL, DataFrames and Datasets Spark Library
• Compare the various APIs – RDD, DataFrames and Datasets
• Lab – Spark DataFrames Transformation & Actions
• Lab – Spark DataFrames Advanced Transformation & Actions
• Demos and Labs

Days3: –

Data Science Overview

• Data Science Process Overview
• Structured and Unstructured Data
• Data Acquisition and Transformation
• Data Analysis and Machine Learning
• Machine Learning Concepts

Machine Learning Overview using Apache Spark

• Introduction to Machine Learning and Data Science
• Machine Learning Spark Library
• Spark Machine Learning – Classification, Regression
• Machine Learning Model building with Spark ML Library
• Demos and Labs

Days4: –

Structured Streaming Overview using Apache Spark

• Need of real time processing
• Streaming Spark Library
• Streaming Query
• Processing and Aggregating Streams
• Data Lake concept
• Spark Streaming examples
• Demos and Labs

Graphix/Graph Frames Overview using Apache Spark

• Need of Graphix/Graph Frames
• Spark Graphx & GraphFrames Library
• Spark Graphx & GraphFrames examples
• Demos and Labs

Training material provided: Yes (Digital format)