Building Data Pipelines in Scala
- October 10, 2017
47 Degrees, in partnership with Trainologic, will be hosting an in-depth, three-day training class called Building Data Pipelines in Scala at The Pioneer Collective in Seattle, Washington.
In this course, students will learn how to build robust, performant, and highly-available data pipelines using Scala and related technologies.
During the course, we will discuss many technical aspects regarding design & implementation considerations around the suggested technology stack. The course will cover Kafka, Flink, Spark, and Akka. The course will focus on the streaming aspect of these technologies.
The course includes hands-on exercises which are crucial to the understanding of the topics.
- Scala Developers
Students taking this course should have:
- Experience with Scala
Introduction to Streaming (2 hours)
- Streaming Design Considerations
- Challenges in Streaming Applications
- Use Cases
- Streaming in Microservices Architecture
Akka Streaming (3 hours)
- Introduction to Akka
- The Actor Model
- Akka Streaming
Spark (6 hours)
- Introduction to Spark
- Resilient-Distributed-Datasets (RDD)
- Overview on Datasets
- Spark Streaming (DStream)
- Structured Streaming
Apache Flink (5 hours)
- Introduction to Flink
- DataSet and DataStream
- Introduction to Apache Kafka
- Kafka as a Spark DataSource
- Kafka as a Flink Source
- Kafka as a Sink
Real-world Streaming (5 hours)
- Event time (timestamps) and Window Aggregations
- Checkpoints and Robustness
- Monitoring and Logging
- Handling State
- Serialization Considerations
- Memory & GC
- Performance Best Practices
Duration: 3 days
About the Instructor
Shimi Bandiel is an expert in Scala, Big Data and JVM technologies. With more than 18 years of development, consulting & training, Shimi has accompanied many development teams during their development phases and help them overcome technical challenges. Shimi is an experienced trainer who delivered more than 400 courses for the last 13 years varying from beginner to expert levels with great feedbacks.
In recent years, Shimi has focused on Scala and Big Data frameworks (Spark & Flink) and helped several organizations develop and tune their solutions.
Want to train your entire team? Contact us to see about a group discount or customized training for your organization.