Apache Flink 101 - the rise of stream processing and beyond

This talk is about some Flink use cases and basic requirements of stream processing, and how Flink fills the gaps and stands out with some of its unique core building blocks, like pipelined execution, native event time support, state support, and fault tolerance.

There is also a highlight of how Flink is going beyond stream processing into areas like unified data processing, enterprise intergration, AI/machine learning  especially online ML, and serverless computation, and how Flink fits with its distinct value.

Links


How to performance-tune Spark applications in large clusters

Omkar Joshi a senoir software engineer at Uber discusses a new Spark ingestion system known as Marmaray. This new system has been designed to ingest billions of Kafka messages at intervals of 30 minutes. 

Links



The Need for Speed – Data Streaming in the Cloud with Kafka®

Running Kafka on Kubernetes is becoming more and more popular. Frank Pientka, Principal Software Architect, Materna Information & Communications SE introduces a setup, used components and recommendations from an own project with Kafka on Kubernetes.He shares the lessons learned from this still evolving field. 

Links




How Traveloka's Runs Cloud-Scale Apache Spark in Production Since 2017

Traveloka's Data Engineering and Data Science team shares how the staff submit their cloud-scale Spark jobs today. The discussion highlights pros/cons, integration of Apache Spark with CI/CD components, Schedulers, Airflow, Key Management Systems (KMS), templates. The journey starts at historic event of a self-managed Spark cluster on-premise, and talk through adoption of AWS EMR, Qubole, Databricks, and Dataproc. How multiple back-end data sets has helped transform Traveloka from meta-search engine to fully integrated On-Line Travel Booking agency, and one of top Indonesian Unicorn startups!

Links


AI in practice: how we help cure diseases using Big Data and AI - Chen Admati @ Intel (Hebrew)

Chen Admati (Head of Intel Pharma Analytics Platform at Intel Corporation), discusses AI in practice and how we help cure diseases using Big Data and AI

Links