At Walmart, while major application software can and does operate in the cloud, stores or any client edge compute cannot avoid the intermittent network events that can create less than ideal availability and performance of the software during those times. This can lead to poor customer experience and/or failed transactions during checkout. Because of Walmart's scale of serving around 265 million customer every week, the comnbined effect on customer experience as well as the loss of revenue is pretty huge.
To overcome the issue between Stores and cloud, Walmart is building and rolling out the next generation of Point of Sale (POS) systems on highly resource constraint edge computing environment using modern service mesh based technologies designed to allow maximum business flexibility, extreme performance and rapid deployment and powered by Kubernetes.
arge-scale streaming processing with Flink and Flink SQL at Alibaba
Source: Seattle Apache Flink Meetup
Also see: Hadoop Weekly 247
Ajinkya Kale of eBay discusses their use of Neo4j as a backend to the AI technology in eBay's virtual shopping assistant: eBay ShopBot. The team discusses how they used Neo4j as a probabilistic graph model to drive conversations based upon their Knowledge Graph. They also touch upon the key learnings for deployment and scalability in Google Cloud Platform, and touch upon the application oriented learnings of using Neo4j for a year in production.
eBay has one of the most mature Enterprise Data Platform’s in the industry with over 200PBs of data stored in Hadoop and Teradata Warehouses. On average 30 TB of transactional and behavioral data is extracted on a daily basis and thousands of metrics are computed, analyzed and monitored for decision making and detecting anomalies. eBay has embarked on an ambitious project to transform the batch oriented ETL processes which could take 24 to 48 hour for metric computation to near real time infrastructure based on Kafka for messaging, Spark Streaming for stream processing and Spark SQL for data preparation.
Salesforce recently invented and deployed a real-time, scalable, terabyte data-level and low false positive personalized anomaly detection system. Anomaly detection on user in-app behavior at terabyte-data scale is extremely challenging because traditional techniques like clustering methods suffer serious production performance issues.
Karthik Deivasigamani from WalMart Labs disusses Storm in retail context: catalog data processing using Kafka, Storm & microservices