Contenu connexe

Similaire à Tranquilizer(20)



  1. Tranquilizer Put your Python functions to REST Albert DeFusco, Ph.D. Data Scientist, Product @ Anaconda, Inc.
  2. Agenda ● What does it mean to productionalize a model? ● What is a REST API? ● Notebooks → REST API with Tranquilizer
  3. After the ML/AI model is trained… Productionize ● Create a Prediction Service to allow easy execution of your model ○ On a website or in an application (like a phone app) ○ Enable others to utilize your model without being a Data Science expert ● Jupyter notebooks are good for Data Scientists ○ Can be challenging for for others to setup and utilize ○ Some people don’t like Jupyter notebooks for production ● Many articles on the web talk about productionalization ● AI/ML models are commonly Deployed as a REST API microservice
  4. Microservices and REST APIs
  5. Microservices and REST APIs ● Microservices are the building blocks of modern web-scale applications ● Separate the components into smaller pieces, even different languages ○ The User Interface ○ The backend database ○ The ML/AI prediction model ● Components communicate over HTTP requests ● Easy to scale and optimize the critical components ● Easy to version and swap-out individual components
  6. Example
  7. Why use REST APIs for ML services? ● Piggy-back on mature web technologies ○ Authentication / authorization ○ Scalability and versioning ○ Compatible with any client that can send HTTP requests REST: REpresentational State Transfer The HTTP commands describe interactions between client (i.e, web browser) and server ● GET: retrieve information ● POST: create a new resource/record ● PUT (UPDATE): update information ● DELETE: delete a resource/record POST is often used for ML/AI prediction REST APIs
  8. Example
  9. How to build REST APIs in Python Do-it-yourself ● Some common frameworks: Flask, Tornado, Django, FastAPI ● Some frameworks may require lots of boilerplate code ● Don’t work natively with Jupyter Notebooks Cloud vendors have developed end-to-end ML platforms to serve REST APIs ● Sage Maker, Seldon, IBM Cloud Pak, and others
  10. The Dream Jupyter Notebooks
  11. Wouldn’t it be great...
  12. Tranquilizer ● First step to Productionize your notebook ○ As little as one line addition ○ Simple command to run ○ Not just for ML/AI models ● Essential skills ○ Functions ○ Type hints really help ○ Docstrings help, too
  13. Tranquilizer What can it do? ● Create and serve REST API endpoints from any Python function ○ Create endpoints from multiple functions in a script file or in a Jupyter Notebook ○ Automatically creates its own documentation ● Inputs and outputs are any data type that can be JSON serialized ○ Python Builtins: strings, numbers, lists, dictionaries ● Extra data types provided by Tranquilizer ○ Datetime, NumPy array, image file, text and binary files ● Easily integrated with other web technologies ○ Docker, Nginx (for reverse proxies, SSL, etc.)
  14. Jupyter → REST API 1. Install Tranquilizer† ● ‘conda install tranquilizer’ or ‘pip install tranquilizer’ 1. Decorate your functions 1. Run the tranquilizer server †Add nbconvert if using notebooks
  15. Demo
  16. ML/AI REST API: Best Practices ● Write fast functions ● Avoid changing “state” of any global variables ○ This is best-practice for REST APIs in general ● Avoid unnecessary cells ○ Especially cells that load a lot of data or take a long time to run ○ All cells are executed when Tranquilizer starts ● Load the model outside of any decorated functions if possible ○ This is even more important for Tensorflow/Keras than for Scikit-Learn
  17. Where do I go from here? ● Containerization ○ Build a Docker image containing the notebook and model files ● Cloud hosting ○ Run the Docker image on a cloud service ○ Secure the container running Tranquilizer behind a reverse proxy (i.e, nginx) ○ Scale the REST API with a load balancer Anything you can do with Flask you can do with Tranquilizer ● The command is ‘tranquilizer <script-or-notebook> --port <port>’
  18. Dockerfile This Dockerfile ● Uses the Miniconda3 base image ● Copies your current directory ● Creates the Conda env from environment.yml ● Runs Tranquilizer over port 8080 See the Github repo for more details
  19. Thank you ● ●