SHOW

Filter (clear filters)

Domains

Companies

Technologies

Functions

Highlights


Overview for uber

Kubernetizing Big Data and ML Workloads at Uber - Mayank Bansal & Min Cai, Uber

Uber relies on Big Data and ML to make business critical decisions such as pricing, trip ETA, etc. Today, those workloads such as Hive and Spark are running on YARN. To save millions of dollars by efficient use of cluster resources, Uber is planning to use Kubernetes to co-locate BigData/ML and micro-service workloads.

This talk will covers the following:
- Learnings of running large-scale BigData/ML on Kubernetes with Peloton
- Colocation of mixed workloads
- Federation across zones
- Feature and API parity with YARN

Links


Only Slightly Bent: Uber’s Kubernetes Migration Journey for Microservices - Yunpeng Liu, Uber

Uber started using docker containers at scale in 2015, and has gone through a few generations of cluster management and service discovery technologies. In early 2019, Uber started working on migration from Mesos to Kubernetes to support secure service mesh and machine learning workloads.
This talk highlights the following: 
- Overview of Uber Compute Infra
- API server benchmark and tweaks
- Custom controller and scheduler logic
- CRI: resource, health check, logging, isolation
- SPIRE and service discovery setup at Uber

Links



How to performance-tune Spark applications in large clusters

Omkar Joshi a senoir software engineer at Uber discusses a new Spark ingestion system known as Marmaray. This new system has been designed to ingest billions of Kafka messages at intervals of 30 minutes. 

Links