Many data scientists are already making heavy usage of the Jupyter ecosystem for analyzing data using interactive notebooks. Apache Toree (incubating) is a Jupyter kernel designed to act as a gateway to Spark by enabling users Spark from standard Jupyter notebooks. This allows users to easily integrate Spark into their existing Jupyter deployments, This allows users to easily move between languages and contexts without needing to switch to a different set of tools. Apache Toree is designed expressly for interactive work. It supports interpreters in Scala, Python, and R. In this talk, I will cover the design of Toree, how it interacts with the Jupyter ecosystem and various ways in which users can extend the functionality of Apache Toree via a powerful plugin system.

1. Apache Toree:
A Jupyter Kernel for Spark
Marius van Niekerk
Apache Toree Contributor

2. Apache Toree
A Jupyter kernel to connect to Spark and create
interactive applications

3. Ecosystem

4. Jupyter
• Open source, interactive data science
and scientific computing across over
40 programming languages.
• Toree is an implementation of the
Jupyter Kernel Protocol

5. Apache Toree History
• Started as ibm/spark-kernel in 2013
• Started Apache incubation in late 2015
• First Apache Releaseis coming soon.

6. Notebooks Dashboards NodeJS
Application
Jupyter
Toree Toree Toree
Kernel
Gateway
EclairJS
Toree
Interactive
Application

7. Compatibility
• Toree 0.1.x supports Spark 1.6.x
• Toree 0.2.x supports Spark 2.x

8. Features
• Kernel languages: Scala, Python, R
• Magics
• Tab completion
• Plugin system

9. Try it out
• Welcome to Spark with Scala:
– https://tmpnb.org
• Docker
– docker run -it --rm -p 8888:8888
jupyter/all-spark-notebook
– This is a very large container ~4gb

11. Examples
• github.com/Lull3rSkat3r/apache-toree-demos
• github.com/apache/incubator-
toree/tree/master/etc/examples/notebooks

12. Extending on top of Toree
• github.com/Brunel-Visualization/Brunel
• github.com/jupyter/declarativewidgets

13. Other Alternatives
• sparkmagic (livy)
• jupyter-scala
• Apache Zeppelin
• spylon-kernel
• databricks

14. Help out
Looking for contributors
l Web - toree.apache.org
l Mailing - dev@toree.incubator.apache.org
l Chat - gitter.im/apache/toree

15. Thank You.
Github: @mariusvniekerk
Twitter: @__mvn__