L7. A developers’ overview of the world of predictive APIs
•
2 j'aime•741 vues
Valencian Summer School 2015 Day 1 Lecture 7 A developers’ overview of the world of predictive APIs Louis Dorard (PAPIs.io) https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (18)
En vedette
En vedette (20)
Similaire à L7. A developers’ overview of the world of predictive APIs
Similaire à L7. A developers’ overview of the world of predictive APIs (20)
Plus de Machine Learning Valencia
Dernier
Dernier (20)
L7. A developers’ overview of the world of predictive APIs
- 1. A developers’ overview of the world of predictive APIs (proprietary, open source, hybrid) Louis Dorard PAPIs.io
- 2. Predictive APIs basics Types of Predictive APIs Deploy own model on 3rd party platform Open source PAPI frameworks What’s missing / almost there
- 5. The two phases of machine learning: • TRAIN a model • PREDICT with a model
- 6. The two methods of predictive APIs: • TRAIN a model • PREDICT with a model
- 7. The two methods of predictive APIs: • model = create_model(dataset) • predicted_output = create_prediction(model, new_input)
- 8. The two methods of predictive APIs: • model = create_model(‘training.csv’) • predicted_output = create_prediction(model, new_input)
- 9. from bigml.api import BigML # create a model api = BigML() source = api.create_source('training_data.csv') dataset = api.create_dataset(source) model = api.create_model(dataset) # make a prediction prediction = api.create_prediction(model, new_input) print "Predicted output value: ",prediction['object'] ['output'] BigML Python
- 10. # from http://scikit-learn.org/stable/tutorial/ basic/tutorial.html from sklearn import svm model = svm.SVC(gamma=0.001, C=100.) from sklearn import datasets digits = datasets.load_digits() model.fit(digits.data[:-1], digits.target[:-1]) model.predict(digits.data[-1]) Scikit Python
- 11. $ curl https://bigml.io/model?$BIGML_AUTH -X POST -H "content-type: application/json" -d '{"dataset": "dataset/50ca447b3b56356ae0000029"}' BigML REST http
- 12. The two methods of predictive APIs: • model = create_model(‘training.csv’) • predicted_output = create_prediction(model, new_input)
- 13. The two methods of predictive APIs: • model = create_model(‘training.csv’) • predicted_output = create_prediction(model, new_input)
- 14. Types of Predictive APIs
- 17. “Is this email important? — Yes/No”
- 18. Generic Predictive APIs: • Google Prediction API • BigML • Datagami • PredicSis • WolframCloud • Amazon
- 19. AMAZON GOOGLE PREDICSIS BIGML ACCURACY 0.862 0.743 0.858 0.790 TRAINING TIME 135s 76s 17s 5s TEST TIME 188s 369s 5s 1s louisdorard.com/blog/machine-learning-apis-comparison
- 21. “Which project does this document relate to? — A/B/C”
- 22. Text classification APIs: • uClassify.com • MonkeyLearn.com • Cortical.io
- 24. “Is this customer going to leave next month? — Yes/No”
- 25. “Is this transaction fraudulent? — Yes/No”
- 26. Problem-specific Predictive APIs: • Churn: ChurnSpotter.io, Framed.io • Fraud detection: SiftScience.com • Lead scoring: Infer.com • Personal assistant (Siri): Wit.ai
- 28. “What is the sentiment of this tweet? — Positive/Neutral/Negative”
- 29. “Is this email spam? — Yes/No”
- 30. Fixed-model Predictive APIs: • Text: Datumbox.com, Semantria.com • Vision: Indico, AlchemyAPI.com • Siri-like: maluuba.com
- 33. Deploy your own model on 3rd party platform
- 34. EXPERIMENT / DEPLOY Find model Use model
- 35. Start with {popular, open source} ML library
- 36. Experiment in automated environment
- 42. Experiment on “ScienceCluster” • Distributed jobs • Collaborative workspace • Serialize chosen model Deploy model as API on “ScienceOps” • Load balancing • Auto scaling • Monitoring (API calls, accuracy)
- 43. 43 • 1 for serving predictions • 1 for running ML experiment (i.e. train and evaluate models on given data)? • 1 for deploying ML models? Your API endpoints
- 46. • “Open source prediction server” • Based on Spark • Expose predictive models as (scalable & robust) APIs
- 47. MLlib
- 49. = PAPI+
- 50. • Train command: “pio train” • API endpoints: • Send (new) data • Send prediction queries
- 52. Deployment • Amazon CloudFormation template → cluster • Manual up/down scaling
- 56. What’s missing / almost there
- 57. 57 • Spearmint: “Bayesian optimization” for tuning parameters → Whetlab → Twitter • Auto-sklearn: “automated machine learning toolkit and drop-in replacement for a scikit-learn estimator” • See automl.org and challenge Open Source AutoML?!
- 58. from sklearn import svm model = svm.SVC(gamma=0.001, C=100.) from sklearn import datasets digits = datasets.load_digits() model.fit(digits.data[:-1], digits.target[:-1]) model.predict(digits.data[-1]) Scikit Python
- 59. from sklearn import svm model = svm.SVC(gamma=0.001, C=100.) from sklearn import datasets digits = datasets.load_digits() model.fit(digits.data[:-1], digits.target[:-1]) model.predict(digits.data[-1]) Scikit Python
- 60. import autosklearn model = autosklearn.AutoSklearnClassifier() from sklearn import datasets digits = datasets.load_digits() model.fit(digits.data[:-1], digits.target[:-1]) model.predict(digits.data[-1]) AutoML Scikit
- 61. 61 • ? Open Source Auto Scaling?!
- 62. AMAZON GOOGLE PREDICSIS BIGML ACCURACY 0.862 0.743 0.858 0.790 TRAINING TIME 135s 76s 17s 5s TEST TIME 188s 369s 5s 1s louisdorard.com/blog/machine-learning-apis-comparison
- 63. 63 • Requirement: • train/test splits on local machine • compute evaluation on local machine • Solutions • adapt bigmler and use local evaluations? • use scikit-learn framework? Automated Benchmark?!
- 64. 64 • Python defacto standard: scikit-learn • “Sparkit-learn aims to provide scikit- learn functionality and API on PySpark. The main goal of the library is to create an API that stays close to sklearn’s." • REST standard: PSI (Protocols & Structures for Inference) • Pretty similar to BigML API! • Implementation for scikit available • Easy benchmarking! Ensembles! API standards?!
- 65. 65 • VM with Jupyter notebooks (Python & Bash) • API wrappers preinstalled: BigML & Google Pred • Notebook for easy setup of credentials • Scikit-learn and Pandas preinstalled • Open source VM provisioning script & notebooks • Search public Snaps on terminal.com: “machine learning” Getting started