Martin Odersky, the creator of Scala, talks about what's next for Scala.

1. Scala Next
SF Scala meetup Dec 8th, 2011

2. Scala Today
2

3. Some adoption vectors:
• Web platforms
• Trading platforms
• Financial modeling
• Simulation
Fast to first product, scalable
afterwards
3

4. Github vs. Stack Overflow
RedMonk: Revisiting the
Dataists Programming
Language Rankings
Typesafe Confidential
4

5. Commercial Adoption
• Scala jobs tripled
in last year
• Now at estimated
100,000
developers
Typesafe Confidential
5

6. (Only 17 months ago!)
Scala New collections
2.8: Package objects
Context bounds
Better implicits
...
6

7. Parallel collections
DelayedInit and App
Scala Faster REPL
Progress on IDEs:
2.9: Eclipse, IntelliJ, Neatbeans,
ENSIME
Better docs
Lots of bug fixes
7

8. Parallel Collections
• Use Java 7 Fork Join framework
• Split work by number of Processors
• Each Thread has a work queue that is split
exponentially. Largest on end of queue
• Granularity balance against scheduling overhead
• On completion threads “work steals” from end of other
thread queues
8

9. ... and its usage
import
java.util.ArrayList;
...
Person[]
people;
Person[]
minors;
Person[]
adults;
{
ArrayList<Person>
minorsList
=
new
ArrayList<Person>();
ArrayList<Person>
adultsList
=
new
ArrayList<Person>();
... in Java:
for
(int
i
=
0;
i
<
people.length;
i++)
(people[i].age
<
18
?
minorsList
:
adultsList)
.add(people[i]);
minors
=
minorsList.toArray(people);
adults
=
adultsList.toArray(people);
}
A function value
An infix method call
... in Scala: val
people:
Array[Person]
val
(minors,
adults)
=
people
partition
(_.age
<
18)
A simple pattern match 9

10. Going Parallel
... in Java: ?
... in Scala: val
people:
Array[Person]
val
(minors,
adults)
=
people.par
partition
(_.age
<
18)
10

11. General Collection Hierarchy
Remove this layer in 2.10?
GenTraversable
GenIterable
Traversable
GenSeq
Iterable ParIterable
Seq ParSeq
11

12. Going Distributed
• Can we get the power of parallel collections to work on
10’000s of computers?
• Hot technologies: MapReduce (Google’s and Hadoop)
• But not everything is easy to fit into that mold
• Sometimes 100’s of map-reduce steps are needed.
• Distributed collections retain most operations, provide a
powerful frontend for MapReduce computations.
• Scala’s uniform collection model is designed to also
accommodate parallel and distributed.
• Projects at Google (Cascade), Berkeley (Spark), EPFL.
12

13. Scala Eclipse IDE
next: Play web framework 2.0
Akka 2.0
Scala 2.10
13

14. Scala Now in RC2
Final expected before the end of
the year.
Eclipse
IDE
14

15. Goals
reliable (no crashes/lock ups)
responsive (never wait when typing)
work with large projects/files
– Scala compiler (80k LOC), 4-5000 LOC/file
– advanced use of the type system:
path-dependent types, self-types, mix-ins

16. Features
Keep it simple
– highlight errors as you type
– completions (including implicits)
– hyperlinking
– project builder (+ dependent projects)
Support mixed Java-Scala projects
– all features should work between Java/Scala sources
JUnit Test Runner should pick up tests
More stuff based on external libraries
– (some) refactoring, code formatter, mark occurrences, structured
selections, show inferred semi-colons

17. Features (3)
based on external libraries
– (some) refactoring
– code formatter
– mark occurrences
– structured selections
– show inferred semi-colons

18. @jonifreeman
Joni Freeman
Latest Scala Eclipse plugin works surprisingly well! Even manages our
mixed Java/Scala project. Kudos to the team! #scala
@esorribas
Eduardo Sorribas
The latest beta of the Scala IDE for eclipse is much better. I'm starting
to like it.
@jannehietamaki
Janne Hietamäki
After years of misery, the Eclipse Scala plugin actually seems to work
quite well.

19. Architecture
Use the full-blown Scala compiler for:
– interactive error highlight, completion, hyperlinking
– turning Scala symbols into Java model elements
Weave the JDT compiler when it needs help
– JDT was NOT meant to be extended

20. Why rely on scalac?
– reuse (type-checker == 1-2 person years)
– consistency
– compiler plugins
Why not?
– SPEED
– (very) tight dependency on the Scala version

21. asynchronous
interruptible
targeted
stop after type-checking
Presentation
Compiler

24. Result is communicated through a SyncVar

25. • All compiler activity happens on PC thread
• compile loaded files when work queue is empty (in the
background)
• Check work queue when type checker reaches safe-points in
the AST
• Drop everything when a file is changed (AskReload)

26. Implementation
26

27. 1 type-checker run / instance --> 100s of type-check runs / minute
– memory leaks
– side-effects/state
– out-of-order and targeted type-checking
needed to improve the compiler
– 2.9.x, 2.10 (trunk)
– what about 2.8?
2.8.2, 2.8.3-SNAPSHOT

28. New: Play Framework 2.0
• Play Framework is an open source web application
framework, inspired by Ruby on Rails, for Java and Scala
• Play Framework 2.0 retains full Java support while moving
to a Scala core and builds on key pieces of the Typesafe
Stack, including Akka middleware and SBT
• Play will be integrated in TypeSafe stack 2.0
• Typesafe will contribute to development and provide
commercial support and maintenance.

29. Roadmap
May 2011 Oct 2011 Q1 2012 Q3 2012
Typesafe Typesafe Typesafe Typesafe
Stack 1.0 Stack 1.1 Stack 2.0 Stack 2.x
Scala 2.9.0 Scala 2.9.1 Scala 2.9.x Scala 2.10
Akka 1.1 Akka 1.2 Akka 2.0 Akka 2.x
Play 2.0 Play 2.x
Slick (DB)

30. 1. New reflection framework
Scala 2.
3.
Reification
type Dynamic
2.10: 4. More IDE improvements: find-
references, debugger,
worksheet.
5. Faster builds
6. SIPs: string interpolation,
simpler implicits.
ETA: Early 2012.
30

31. New in Scala 2.10: Dynamic
Type Dynamic bridges the gap between static and dynamic typing.
Method calls get translated to applyDynamic
Great for interfacing with dynamic languages (e.g. JavaScript)
class JS extends Dynamic {
def applyDynamic(methName: String, args: Any*): Any = {
println("apply dynamic "+methName+args.mkString("(", ",", ")"))
}
}
val x = new JS
x.foo(1) // à x.applyDynamic( foo , 1)
x.bar // à x.applyDynamic( bar )
31

32. Proposed for Scala 2.10:
SIP 11: String interpolation
Idea: Instead of
Bob
is
+
n
+
years
old
write:
s Bob
is
$n
years
old
which gets translated to
new
StringContext( Bob
is ,
years
old ).s(n)
Here, s is a library-defined method for string interpolation.
32

33. This can be generalized to other string processors besides s:
xml
<body>
<a
href
=
some
link >
${linktext}
</a>
</body>
scala
scala.concurrent.transaction.withinTransaction
{
(implicit
currentTransaction:
Transaction)
=>
$expr
}
33

34. Proposed for Scala 2.10:
SIP 12: Uncluttering control
Should be able to write:
if
x
<
0
then
–x
else
x
while
x
>
0
do
{
println(x);
x
-­‐=
1
}
for
x
<-­‐
xs
do
println(x)
for
x
<-­‐
xs
yield
x
*
x
34

35. Proposed for Scala 2.10:
SIP 13: Implicit classes
Variation:
Add @inline
to class def to get speed of extension methods.
35

36. New in Scala 2.10: Reflection
Previously: Needed to use Java reflection,
no runtime info available on Scala s types.
Now you can do:
36

37. (Bare-Bones) Reflection in Java
Why not add some
meaningful operations?
Need to write essential
parts of a compiler
(hard).
Need to ensure that
both compilers agree
(almost impossible).
Want to know whether type A conforms to B?
Write your own Java compiler!
37

38. How to do Better?
• Problem is managing dependencies between compiler and
reflection.
• Time to look at DI again.
Dependency Injection
• Idea: Avoid hard dependencies to specific classes.
• Instead of calling specific classes with new, have someone else do
the wiring.
38

39. Using Guice for Dependency Injection
(Example by Jan Kriesten)
39

40. ... plus some Boilerplate
40

41. Dependency Injection in Scala
Components are
classes or traits
Requirements are
abstract values
Wiring by implementing
requirement values
But what about cyclic dependencies?
41

42. The Cake Pattern
Components are traits
Wiring by mixin
composition
Requirements are
types of this
42

43. Cake Pattern in the Compiler
The Scala compiler uses the cake pattern for everything
Here s a schema:
(In reality there are about ~20 slices in the cake.)
43

44. Towards Better Reflection
Can we unify the core parts of the compiler and reflection?
Compiler Reflection
Different requirements: Error diagnostics, file access, classpath
handling - but we are close!
44

45. Compiler Architecture
Problem: This exposes way too much detail!
reflect.internal.Universe
nsc.Global
(scalac) reflect.runtime.Mirror
45

46. Complete Reflection Architecture
reflect.api.Universe
/
Cleaned-up facade:
reflect.mirror
reflect.internal.Universe
Full implementation:
nsc.Global
(scalac) reflect.runtime.Mirror
46

47. How to Make a Facade
The Facade
Interfaces are not enough!
The Implementation
47

48. Conclusion
Scala is a very regular language when it comes to composition:
1. Everything can be nested:
– classes, methods, objects, types
2. Everything can be abstract:
– methods, values, types
3. The type of this can be declared freely, can thus express
dependencies
4. This gives great flexibility for SW architecture, allows us to attack
previously unsolvable problems.
48

49. Going further: Parallel DSLs
Mid term, research project: How do we keep tomorrow s
computers loaded?
– How to find and deal with 10000+ threads in an
application?
– Parallel collections and actors are necessary but not
sufficient for this.
Our bet for the mid term future: parallel embedded DSLs.
– Find parallelism in domains: physics simulation, machine
learning, statistics, ...
Joint work with Kunle Olukuton, Pat Hanrahan @ Stanford.
EPFL side funded by ERC.
49

50. EPFL / Stanford Research
Scientific Virtual Personal Data
Applications Engineering Worlds Robotics informatics
Domain
Physics Probabilistic Machine
Specific Rendering Scripting Learning
Languages (Liszt) (RandomT) (OptiML)
Domain Embedding Language (Scala)
Polymorphic Embedding Staging Static Domain Specific Opt.
DSL
Infrastructure
Parallel Runtime (Delite, Sequoia, GRAMPS)
Dynamic Domain Spec. Opt. Task & Data Parallelism Locality Aware Scheduling
Hardware Architecture
Heterogeneous
OOO Cores SIMD Cores Threaded Cores Specialized Cores
Hardware
Programmable Scalable Isolation & On-chip Pervasive
Hierarchies Coherence Atomicity Networks Monitoring
50

51. Example: Liszt - A DSL for Physics
Simulation
Combustion
Turbulence
Fuel injection
Transition Thermal
• Mesh-based
• Numeric Simulation
• Huge domains Turbulence
– millions of cells
• Example: Unstructured Reynolds-averaged Navier
Stokes (RANS) solver
51

52. Liszt as Virtualized Scala
val // calculating scalar convection (Liszt)
val Flux = new Field[Cell,Float] AST
val Phi = new Field[Cell,Float]
val cell_volume = new Field[Cell,Float]
val deltat = .001
...
untilconverged {
for(f <- interior_faces) {
val flux = calc_flux(f)
Flux(inside(f)) -= flux
Flux(outside(f)) += flux
}
for(f <- inlet_faces) { Optimisers Generators
Flux(outside(f)) += calc_boundary_flux(f)
} …
for(c <- cells(mesh)) {
Phi(c) += deltat * Flux(c) /cell_volume Schedulers
(c)
} …
for(f <- faces(mesh))
Flux(f) = 0.f Hardware
}
DSL Library GPU, Multi-Core, etc
52

53. Follow us on twitter: @typesafe
akka.io
scala-lang.org
typesafe.com
scala-lang.org
53