Scaling machine learning as a service at Uber — Li Erran Li at #papis2016
•
4 j'aime•1,600 vues
Machine learning as a service (MLaS) is imperative to the success of many companies as many internal teams and organizations need to gain business intelligence from big data. Building a scalable MLaS in a very challenging problem. In this paper, we present the scalable MLaS we built for a company that operates globally. We focus on several scalability challenges and our technical solutions. Video at https://www.youtube.com/watch?v=MpnszJ_3Ong Couldn't attend PAPIs '16? Get access to the other presentations' slides and videos at https://gumroad.com/products/fehon/
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Scaling machine learning as a service at Uber — Li Erran Li at #papis2016
Similaire à Scaling machine learning as a service at Uber — Li Erran Li at #papis2016 (20)
Plus de PAPIs.io
Plus de PAPIs.io (20)
Dernier
Dernier (20)
Scaling machine learning as a service at Uber — Li Erran Li at #papis2016
- 5. Use Case: UberEATS ETD Prediction 5
- 7. HADOOP / YARN (Batch)
- 8. HADOOP / YARN (Batch) Hive Feature Store
- 9. NETWORK (Realtime) HADOOP / YARN (Batch) Cassandra Feature Store Hive Feature Store
- 11. Real-time prediction Training Use Case: UberEATS ETD ML Pipeline Hive 11 Feature store Model Training Model UberEATS App Model Performance ETD
- 12. Problems • Hard to figure out good features • Hard to build the pipelines to generate features • Can’t compute some features in real time Solution: DSL and Feature Store ● Database of curated and crowd-sourced features ● Make it easy to use and transform these features in ML projects ● Make it easy to discover new useful features ● Batch and realtime serving
- 13. Data Pipeline For Predictions Feature DSL Transformed Features Basis Features ML Model PredictionsData Lake Spark or SQL
- 14. Data Pipeline For Predictions w/ Feature Palette Feature Store Feature DSL Transforme d Features Basis Features ML Model PredictionsData Lake Spark or SQL
- 15. Use Case: UberEATS ETD Model Details 15 Feature store Model: GBT RegressionUberEATS App ETD ● restaurant features ○ location, avg prep-time, avg delivery time, avg demand during lunch ... ● contextual features ○ time of day, day of week, ... ● order features ○ #items, total cost, ... ● near real-time features ○ info about the past N orders, ... ● ... ● Feature store provides aggregate features for real-time prediction ○ These features are time-consuming to compute in real-time
- 16. Problem ● Often you want to train a model per city ● But hard to train and manage 400+ models for a project Solution ● Let users define partitioning scheme ● Automatically train model per partition ● Manage and deploy as single logical model
- 17. 1. Define partition scheme
- 18. 2. Make train / test split
- 19. 3. Keep same split for each level
- 20. M M M M M M M M M 4. Train model for every node
- 21. M M M M M M M M 5. Prune bad models
- 22. M M M M M M M M 6. At prediction time, use best model for each node
- 24. Use Case: UberEATS ETD Prediction Performance 24 ● Partitioned GBDT Regression Model ● Latency (measured from client) ○ p50: 7ms ○ p95: 15ms ○ p99: 20ms
- 25. Conclusion ● We present a scalable ML as a service system ● We focus on the scalability challenges and solutions ○ Feature store key to enable aggregate features for real-time prediction ■ Same API to access feature store for both batch training and real-time prediction ○ Partitioned models greatly simplifies model management and selection ■ Per city model performance often worse than global model ○ Scalable low latency real-time prediction service enables interactive user experiences ■ Load balancing across containers without global state ■ Fast one button deployment ■ Hot swap model upgrade