SHOW

Filter (clear filters)

Domains

Companies

Technologies

Functions

Highlights


Overview for stream processing

The Need for Speed – Data Streaming in the Cloud with Kafka®

Running Kafka on Kubernetes is becoming more and more popular. Frank Pientka, Principal Software Architect, Materna Information & Communications SE introduces a setup, used components and recommendations from an own project with Kafka on Kubernetes.He shares the lessons learned from this still evolving field. 

Links



How Traveloka's Runs Cloud-Scale Apache Spark in Production Since 2017

Traveloka's Data Engineering and Data Science team shares how the staff submit their cloud-scale Spark jobs today. The discussion highlights pros/cons, integration of Apache Spark with CI/CD components, Schedulers, Airflow, Key Management Systems (KMS), templates. The journey starts at historic event of a self-managed Spark cluster on-premise, and talk through adoption of AWS EMR, Qubole, Databricks, and Dataproc. How multiple back-end data sets has helped transform Traveloka from meta-search engine to fully integrated On-Line Travel Booking agency, and one of top Indonesian Unicorn startups!

Links


Apache Beam meetup 7 at Datatonic: Beam at Lyft + datalake using Beam + schemas

See how Lyft and Datatonic are using Apache Flink, Apache beam and python in stream processing, machine learning and analytics.

Links