SHOW

Filter (clear filters)

Companies

Technologies

Functions

Domains

Highlights


Overview for kafka




Event Driven Microservices with Kafka & Evolution of the Data Pipeline in Agoda

Agoda is one of the best known travel booking sites in Asia. It is also one of the most sophisticated Kafka users in Asia; processing 400 billion events daily (175 TB) using 230 Kafka brokers across over 20 kafka clusters.
In this talk, Shaun Sit Agoda's development manager focuses on the Evolution of Data Pipeline over the years, the architecture, challenges and lessons learned, design decisions and future roadmap as Agoda continues to scale the pipeline to serve the business needs.

Links


Big Data, Fast Data @ PayPal

PayPal has over 7PB of data in Apache Kafka, so they've got some good experience with building fast data products. In this presentation Sid Anand the Chief Data Engineer at PayPal covers their architecture, like how they use change data capture and why you should use Apache Avro for your data in Kafka.

Links


Apache Kafka® Delivers a Single Source of Truth for The New York Times

The New York Times has over 3.6 million paid print and digital subscriptions. Lean how New York Times changed its its infrastructure at the core to keep up with the new expectations of the digital age and its consumers.  Every piece of content that has been published by The New York Times throughout the past 166 years and counting is stored in Apache Kafka. 

In this talk, Boerge Svingen The New York Times' Director of Engineering highlights the following: 
-An overview of what the publishing infrastructure used to look like 
-Deep dive into the log-based architecture of The New York Times’ Publishing Pipeline 
-The schema, monolog and skinny log used for storing articles 
-Challenges and lessons learned 
-Answers live questions submitted by the audience 

Links