Apache Hadoop

Before enrolling into the course, students must have a brief idea on the syllabus. Below mentioned are some of the important topics that are covered in the entire course.

MODULE I – HADOOP BASICS

---

7 VIDEOS WHICH CONSIST OF

1. Hadoop Stack Basics
2. The Apache Framework: Basic Modules
3. Hadoop Distributed File System (HDFS)
4. The Hadoop “Zoo”
5. Hadoop Ecosystem Major Components
6. Exploring the Cloudera VM: Hands-On Part 1
7. Exploring the Cloudera VM: Hands-On Part 2

4 READINGS

1. Apache Hadoop Ecosystem
2. Lesson 1 Slides (PDF)
3. Hardware & Software Requirements
4. Lesson 2 Slides – Cloudera VM Tour

1 PRACTICE EXERCISE

1. Basic Hadoop Stack

MODULE II – INTRODUCTION TO HADOOP STACK

---

10 VIDEOS

1. Overview of the Hadoop Stack
2. The Hadoop Distributed File System (HDFS) and HDFS
3. MapReduce Framework and YARN
4. The Hadoop Execution Environment
5. YARN, Tez, and Spark
6. Hadoop Resource Scheduling
7. Hadoop-Based Applications
8. Introduction to Apache Pig
9. Introduction to Apache HIVE
10. Introduction to Apache HBASE

6 READINGS

1. Hadoop Basics – Lesson 1 Slides
2. Lesson 2: Hadoop Execution Environment – Slides
3. Lesson 3: Hadoop-based Applications Overview – All Slides
4. Command list for Applications Slide
5. Tips to handle service connection errors
6. References for Application

3 PRACTICE EXERCISE

1. Overview of Hadoop Stack
2. Hadoop Execution Environment
3. Hadoop Applications

MODULE III – INTRODUCTION TO HADOOP DISTRIBUTED FILE SYSTEM

---

9 VIDEOS

1. Overview of HDFS Architecture
2. The HDFS Performance Envelope
3. Read/Write Processes in HDFS
4. HDFS Tuning Parameters
5. HDFS Performance and Robustness
6. Overview of HDFS Access, APIs, and Applications
7. HDFS Commands
8. Native Java API for HDFS
9. REST API for HDFS

5 READINGS

1. Lesson 1: Introduction to HDFS – Slides
2. HDFS references
3. Lesson 2: HDFS Performance and Tuning – Slides
4. HDFS Access, APIs
5. Lesson 3: HDFS Access, APIs, Applications – Slides

3 PRACTICE EXERCISES

1. HDFS Architecture
2. HDFS performance, tuning, and robustness
3. Accessing HDFS

MODULE IV – INTRODUCTION TO MAP/ REDUCE

---

9 VIDEOS

1. Introduction to Map/Reduce
2. The Map/Reduce Framework
3. A MapReduce Example: Wordcount in detail
4. MapReduce: Intro to Examples and Principles
5. MapReduce Example: Trending Wordcount
6. MapReduce Example: Joining Data
7. MapReduce Example: Vector Multiplication
8. Computational Costs of Vector Multiplication
9. MapReduce Summary

3 READINGS

1. Lesson 1: Introduction to MapReduce – Slides.
2. A note on debugging map/reduce programs.
3. Lesson 2: MapReduce Examples and Principles – Slides

1 PRACTICE EXERCISE

1. Lesson 1 Review

MODULE V – SPARK

---

10 VIDEOS

1. Introduction to Apache Spark
2. Architecture of Spark
3. Resilient Distributed Datasets
4. Spark Transformations
5. Wide Transformations
6. Directed Acyclic Graph (DAG) Scheduler
7. Actions in Spark
8. Memory Caching in Spark
9. Broadcast Variables
10. Accumulators

4 READINGS

1. Setup PySpark on the Cloudera VM
2. Lesson 1: Intro to Apache Spark – Slides
3. Lesson 2: RDD and Transformations – Slides
4. Lesson 3: Scheduling, Actions, Caching – Slides

3 PRACTICE PAPERS

1. Spark Lesson 1
2. Spark Lesson 2
3. Spark Lesson 3