We are developing solutions that make it easier to work with data on Hadoop regardless the skill level of the user (whether a developer, an analyst or something else.) The core engine we use and the user-interface have been in research and under development for several years and we are only now starting to “expose” it to the world at large. To give you a sense for our approach, we will provide a variety of demos and discuss the underlying technology to demonstrate the time savings when working with Hadoop. Speaker: Shevek, CompilerWorks Shevek is an expert programmer with a strong interest in parallel and distributed systems. He has worked on cutting edge research in compilers and language design, algorithmic optimization, systems and security. He is capable of maintaining a very straight face under questioning on topics including “Why is our printer playing ‘happy birthday’?” or “What is that message doing on the side of that building?” He received a Doctorate in Computing on the Formalization of Protection Systems from the University of Bath, England. He also holds a Masters in Pure Mathematics and an epee.

1. Compiler Works
Our Goal: To make working with
data on Hadoop easy
Who are we? A group of backend engineers attempting to add
a UI to some technology we've built.
January 2015

2. Compiler Works Development Process Flow
Jan 2015 2
Write Code
(1 second)
Compile Code
(5 minutes)
Test Code
(1 hour)
Integrate Code
(1 day – 1 week)
Deploy to Customer
(3 months)

3. Compiler Works Problems
Jan 2015 3
Write Code
(1 second)
Compile Code
(5 minutes)
Test Code
(1 hour)
Integrate Code
(1 day – 1 week)
Deploy to Customer
(3 months)
IDE highlights bug(s)
with static analysis
Code leaves
developer desktop
Say prayers
Kind of expensive
Runtime errors
detected

4. Compiler Works Writing Code with Fast Feedback?
Jan 2015 4
DEMO

5. Compiler Works What did we solve?
(multi-stage compiler)
Jan 2015 5

6. Compiler Works What did we solve?
(algebraic optimization)
Jan 2015 6
• Join A, B, C, D
• These are the object plans
– they are leaf nodes in the algebraic tree
– they are themselves trees

7. Compiler Works What did we solve?
(scheduling)
Jan 2015 7
• In one cycle, a CPU can execute:
– Ten adds
– Four multiplies
– Two divides
• Schedule this:

8. Compiler Works More Demo
Jan 2015 8
• Complex joins
• Optimizations
• e.t.c.

9. Compiler Works Process Management
Jan 2015 9
• Development
• Experimentation
• Production (export)
• Maintenance (import)
• Re-experimentation
• Reproduction
• Re-export

10. Compiler Works More Demo
Jan 2015 10
• Export
• What else are you interested in?

11. Compiler Works Questions
Jan 2015 11
• I will answer on stage (if time)
• I will answer afterwards (if time)
• Email: info@compilerworks.com