https://github.com/AlbertDeFusco/python-summit-2021
Tranquilizer
Put your Python functions to REST
Albert DeFusco, Ph.D.
Data Scientist, Product @ Anaconda, Inc.

https://github.com/AlbertDeFusco/python-summit-2021
Agenda
● What does it mean to productionalize a model?
● What is a REST API?
● Notebooks → REST API with Tranquilizer

https://github.com/AlbertDeFusco/python-summit-2021
After the ML/AI model is trained… Productionize
● Create a Prediction Service to allow easy execution of your model
○ On a website or in an application (like a phone app)
○ Enable others to utilize your model without being a Data Science expert
● Jupyter notebooks are good for Data Scientists
○ Can be challenging for for others to setup and utilize
○ Some people don’t like Jupyter notebooks for production
● Many articles on the web talk about productionalization
● AI/ML models are commonly Deployed as a REST API microservice

https://github.com/AlbertDeFusco/python-summit-2021
Microservices and REST APIs
https://towardsdatascience.com/launch-your-own-rest-api-using-flask-python-in-7-minutes-c4373eb34239

https://github.com/AlbertDeFusco/python-summit-2021
Microservices and REST APIs
● Microservices are the building blocks of modern web-scale applications
● Separate the components into smaller pieces, even different languages
○ The User Interface
○ The backend database
○ The ML/AI prediction model
● Components communicate over HTTP requests
● Easy to scale and optimize the critical components
● Easy to version and swap-out individual components

https://github.com/AlbertDeFusco/python-summit-2021
Example

https://github.com/AlbertDeFusco/python-summit-2021
Why use REST APIs for ML services?
● Piggy-back on mature web
technologies
○ Authentication / authorization
○ Scalability and versioning
○ Compatible with any client that can
send HTTP requests
REST: REpresentational State Transfer
The HTTP commands describe interactions
between client (i.e, web browser) and server
● GET: retrieve information
● POST: create a new resource/record
● PUT (UPDATE): update information
● DELETE: delete a resource/record
POST is often used for ML/AI prediction REST
APIs

https://github.com/AlbertDeFusco/python-summit-2021
Example

https://github.com/AlbertDeFusco/python-summit-2021
How to build REST APIs in Python
Do-it-yourself
● Some common frameworks: Flask, Tornado, Django, FastAPI
● Some frameworks may require lots of boilerplate code
● Don’t work natively with Jupyter Notebooks
Cloud vendors have developed end-to-end ML platforms to serve REST APIs
● Sage Maker, Seldon, IBM Cloud Pak, and others

https://github.com/AlbertDeFusco/python-summit-2021
The Dream
https://towardsdatascience.com/launch-your-own-rest-api-using-flask-python-in-7-minutes-c4373eb34239
Jupyter Notebooks

https://github.com/AlbertDeFusco/python-summit-2021
Wouldn’t it be great...

https://github.com/AlbertDeFusco/python-summit-2021
Tranquilizer
● First step to Productionize your notebook
○ As little as one line addition
○ Simple command to run
○ Not just for ML/AI models
● Essential skills
○ Functions
○ Type hints really help
○ Docstrings help, too
https://github.com/ContinuumIO/tranquilizer

https://github.com/AlbertDeFusco/python-summit-2021
Tranquilizer
What can it do?
● Create and serve REST API endpoints from any Python function
○ Create endpoints from multiple functions in a script file or in a Jupyter Notebook
○ Automatically creates its own documentation
● Inputs and outputs are any data type that can be JSON serialized
○ Python Builtins: strings, numbers, lists, dictionaries
● Extra data types provided by Tranquilizer
○ Datetime, NumPy array, image file, text and binary files
● Easily integrated with other web technologies
○ Docker, Nginx (for reverse proxies, SSL, etc.)

https://github.com/AlbertDeFusco/python-summit-2021
Jupyter → REST API
1. Install Tranquilizer†
● ‘conda install tranquilizer’ or ‘pip install tranquilizer’
1. Decorate your functions
1. Run the tranquilizer server
†Add nbconvert if using notebooks

https://github.com/AlbertDeFusco/python-summit-2021
Demo

https://github.com/AlbertDeFusco/python-summit-2021
ML/AI REST API: Best Practices
● Write fast functions
● Avoid changing “state” of any global variables
○ This is best-practice for REST APIs in general
● Avoid unnecessary cells
○ Especially cells that load a lot of data or take a long time to run
○ All cells are executed when Tranquilizer starts
● Load the model outside of any decorated functions if possible
○ This is even more important for Tensorflow/Keras than for Scikit-Learn

https://github.com/AlbertDeFusco/python-summit-2021
Where do I go from here?
● Containerization
○ Build a Docker image containing the notebook and model files
● Cloud hosting
○ Run the Docker image on a cloud service
○ Secure the container running Tranquilizer behind a reverse proxy (i.e, nginx)
○ Scale the REST API with a load balancer
Anything you can do with Flask you can do with Tranquilizer
● The command is ‘tranquilizer <script-or-notebook> --port <port>’

https://github.com/AlbertDeFusco/python-summit-2021
Dockerfile
This Dockerfile
● Uses the Miniconda3 base image
● Copies your current directory
● Creates the Conda env from
environment.yml
● Runs Tranquilizer over port 8080
See the Github repo for more details

https://github.com/AlbertDeFusco/python-summit-2021
Thank you
● https://github.com/ContinuumIO/tranquilizer
● https://linkedin.com/in/albertdefusco