Cloudera Developer Training for Apache Spark

3 Days

Description

Cloudera University’s three-day training course for Apache Spark enables participants to build complete, unified big data applications combining batch, streaming, and interactive analytics on all their data. With Spark, evelopers can write sophisticated parallel applications to execute faster decisions, better decisions, and real-time actions, applied to a wide variety of use cases, architectures, and industries.

Skills Gained

Through instructor-led discussion and interactive, hands-on exercises, participants will navigate the Hadoop ecosystem, learning topics such as:

  • Using the Spark shell for interactive data analysis
  • The features of Spark’s Resilient Distributed Datasets
  • How Spark runs on a cluster
  • How Spark parallelizes task execution
  • Writing Spark applications
  • Processing streaming data with Spark

Who Can Benefit

  • This course is best suited to developers and engineers with prior knowledge and experience with Hadoop.

Advance Your Ecosystem Expertise

Apache Spark is the next-generation successor to MapReduce. Spark is a powerful, opensource processing engine for data in the Hadoop cluster, optimized for speed, ease of use, and sophisticated analytics. The Spark framework supports streaming data processing and complex, iterative algorithms, enabling applications to run up to 100x aster than traditional Hadoop MapReduce programs.

No Upcoming Public Classes

There are currently no public events available for this course. However, you can submit a request for a new date and we will try our best to get you into a Cloudera Developer Training for Apache Spark class.

Private Training Available
No date scheduled, don’t see a date that works for you or looking for a private training event, please call 651-905-3729 or submit a request for further information here.
request a private session or new date

Course Overview

  • Introduction to Spark
    • What is Spark?
    • Review: From Hadoop MapReduce to Spark
    • Review: HDFS
    • Review: YARN
    • Spark Overview
  • Spark Basics
    • Using the Spark Shell
    • RDDs (Resilient Distributed Datasets)
    • Functional Programming in Spark
  • Working with RDDs in Spark
    • Creating RDDs
    • Other General RDD Operations
  • Aggregating Data with Pair RDDs
    • Key-Value Pair RDDs
    • Map-Reduce
    • Other Pair RDD Operations
  • Writing and Deploying Spark Applications
    • Spark Applications vs. Spark Shell
    • Creating the SparkContext
    • Building a Spark Application (Scala and Java)
    • Running a Spark Application
    • The Spark Application Web UI
    • Hands-On Exercise: Write and Run
    • Spark Application
    • Configuring Spark Properties
    • Logging
  • Parallel Processing
    • Review: Spark on a Cluster
    • RDD Partitions
    • Partitioning of File-based RDDs
    • HDFS and Data Locality
    • Executing Parallel Operations
    • Stages and Tasks
  • Spark RDD Persistence
    • RDD Lineage
    • RDD Persistence Overview
    • Distributed Persistence
  • Basic Spark Streaming
    • Spark Streaming Overview
    • Example: Streaming Request Count
    • DStreams
    • Developing Spark Streaming Applications
  • Advanced Spark Streaming
    • Multi-Batch Operations
    • State Operations
    • Sliding Window Operations
    • Advanced Data Sources
  • Common Patterns in Spark Data Processing
    • Common Spark Use Cases
    • Iterative Algorithms in Spark
    • Graph Processing and Analysis
    • Machine Learning
    • Example: k-means
  • Improving Spark Performance
    • Shared Variables: Broadcast Variables
    • Shared Variables: Accumulators
    • Common Performance Issues
    • Diagnosing Performance Problems
  • Spark SQL and DataFrames
    • Spark SQL and the SQL Context
    • Creating DataFrames
    • Transforming and Querying DataFrames
    • Saving DataFrames
    • DataFrames and RDDs
    • Comparing Spark SQL, Impala and Hive-on-Spark
  • Conclusion

No Upcoming Public Classes

There are currently no public events available for this course. However, you can submit a request for a new date and we will try our best to get you into a Cloudera Developer Training for Apache Spark class.

Private Training Available
No date scheduled, don’t see a date that works for you or looking for a private training event, please call 651-905-3729 or submit a request for further information here.
request a private session or new date

Prerequisites

Course examples and exercises are presented in Python and Scala, so knowledge of one of these programming languages is required. Basic knowledge of Linux is assumed.

No Upcoming Public Classes

There are currently no public events available for this course. However, you can submit a request for a new date and we will try our best to get you into a Cloudera Developer Training for Apache Spark class.

Private Training Available
No date scheduled, don’t see a date that works for you or looking for a private training event, please call 651-905-3729 or submit a request for further information here.
request a private session or new date