During the tranSMART Annual Meeting 2015, a hackathon will take place for developers to work an a few proof of concepts innovations around tranSMART, also preparing for the future 2.0 version of the platform. There will be two main topics during this hackathon, one catering to backend and one more to frontend developers. The topics are: building a POC around using SparkR on Amazon EC2 as a computational backend for tranSMART 1.3 improving the visual analytics in tranSMART, by updating or adding analytics workflows in the SmartR plugin The reason for choosing these topics are: For SparkR: Spark is the most active project in data science at the moment, with a lot of innovation and big names behind it. It makes sense to explore how we can optimally leverage this in the next version of the tranSMART platform. For this hackathon, a specific proposal has been prepared to use SparkR on top of the tranSMART core API to take advantage of the parallelization and lazy execution capabilities of Spark. For Visual Analytics: the analytics in tranSMART are useful to get a quick overview of the data available in the platform, and one of the most visible and useful capabilities for early adopters to understand the value of the platform. The recently developed SmartR plugin (presented elsewhere in the meeting) provides already a few interactive analytics workflows, improving those and adding new ones is identified as a good opportunity for the hackathon. Ideas for specific analytics workflows to work on are most welcome, and of course we are looking forward to welcome again both beginners and tranSMART developer ninja's in this years' sessions!

1. Hackathon Introduction
TRANSMART ANNUAL MEETING 2015
AMSTERDAM, OCTOBER 19, 2015
Kees van Bochove, Chair Architecture Working Group @ tranSMART Foundation | CEO @ The Hyve

2. 2

3. 3
Hackathon Topics
u Building a POC around using SparkR on Amazon EC2 as
a computational backend for tranSMART 1.3
u Improving the visual analytics in tranSMART, by updating
or adding analytics workflows in the SmartR plugin

4. 4
Apache Spark
u Largest and most active open source project in data
science as of this year
u Seen by many as a ‘replacement’ for Hadoop
[MapReduce] in the big data area
u Implements lessons learned from Hadoop; built from the
ground up as a framework support data scientists
u Core concept: in memory datasets & lazy evaluation

5. 5
SparkR architecture

6. 6
SparkR architecture

7. 7
Hackathon Goal: SparkR integration
u Task: Integrate Spark with tranSMART database, via the
implementation of Spark RDD interface in tranSMART core API
u Goal: Demonstrate scalability on compute side (scalability on
database remains limited because of relational database
paradigm)
u Benefit: Can use Spark compatible applications on top of
tranSMART (e.g. machine learning, big data analytics tools etc.)

8. 8
Current Architecture

9. 9
Goal architecture

10. 10
SparkR architecture

11. 11
SmartR
u Plugin for tranSMART
1.3 written by
Sascha Hertzinger,
Uni. Luxembourgh
for IMI eTRIKS
u Currently being
extended by The
Hyve and Sanofi

12. 12
Current SmartR analytics
u Boxplot
u Correlation Analysis
u Heatmap
u Timeline Analysis
u ? Your ideas
u TODO: add screenshots

13. 13
Hackathon Goal: Visual Analytics
u Improve existing analytics workflows
u e.g. heatmap
u Add new workflows
u D3.js library for building visualizations