GraalVM Workshop for Scala in Tokyo slides by Thomas Wuerthinger on 12/21 2019.

1. GraalVM
Thomas Wuerthinger
GraalVM Project Lead
Oracle Labs
@thomaswue
December 2019

2. Safe harbor statement
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing decisions.
The development, release, timing, and pricing of any features or functionality described for Oracle’s
products may change and remains at the sole discretion of Oracle Corporation.
GraalVM Native Image technology (including SubstrateVM) is Early Adopter technology. It is
available only under an early adopter license and remains subject to potentially significant further
changes, compatibility testing and certification.
2 © 2019 Oracle

3. ~8 years ago, we started a little research project…
© 2019 Oracle3

4. Java
C
C++
Python
C#
VB.NET
JavaScript
Assembly
languagePHP
PerlRubyVBSwift
R
Objective-C
Go
MATLAB
Delphi/Object
Pascal
PL/SQL
Scratch
Others
Programming Language Popularity
(TOP 20 Languages From May 2018 Tiobe INDEX)
© 2019 Oracle4

5. GraalVM will be the Universal VM
© 2019 Oracle
ubiquitous across the cloud stack
1. run programs more efficient
2. make developers more productive
5

6. What is Graal VM?
• Drop-in replacement for Oracle Java 8 (and soon Java 11)
• Run your Java application faster
• Ahead-of-time compilation for Java
• Create standalone binaries with low footprint
• High-performance JavaScript, Python, Ruby, R, ...
• The first VM for true polyglot programming
• Implement your own language or DSL
© 2019 Oracle6

7. Native Image
© 2019 Oracle7

8. © 2019 Oracle8
FREE on Oracle Cloud!

9. GraalVM Open Source
© 2019 Oracle9
Open Source LOC actively maintained by GraalVM team
Total: 3,640,000 lines of code

10. GraalVM Vision:
Abstractions should be without performance regret!
© 2019 Oracle10

11. GraalVM JIT Performance: Renaissance.dev
1.02
1.27
1.53
1.1
1.36
1.37
1.17
1.49
1.14
1.14
1.96
1.03
1.03
3.08
1.35
1.1
1.12
1.13
1.66
1.17
1.59
1.32
0.97
0.99
1
1
1.09
1.2
0.79
0.93
1.03
1.09
0.94
0.83
0.97
2.78
1.02
0.98
1.12
0.98
1.16
1.04
1.09
1.06
0
0.5
1
1.5
2
2.5
3
3.5
akka-uct
alschi-squaredb-shootout
dec-tree
dotty
finagle-chirperfinagle-httpfj-km
eans
future-geneticgauss-m
ix
log-regressionm
ovie-lensnaive-bayes
neo4j-analyticspage-rank
philosophersrx-scrabble
scrabblestm
-bench7
stream
s-m
nem
onicsgeom
ean
SpeedupvsJDK8
EE/C2 CE/C2
© 2019 Oracle11

12. More Benchmarks…
Optimizing for too few benchmarks is like overfitting a machine learning
algorithm.
Therefore we started together with academic collaborators
https://renaissance.dev
All benchmark data can be interesting; careful with conclusions though.
© 2019 Oracle12

13. Java Flight Recorder Compilation Information
© 2019 Oracle13

14. GraalVM Native Image
© 2019 Oracle14

15. native-image MyMainClass
./mymainclass
JIT AOT
java MyMainClass
© 2019 Oracle15

16. Native Image Architecture
© 2019 Oracle
Ahead-of-Time
Compilation
Application
Libraries
JDK
Substrate VM
Points-to Analysis
Run Initializations
Heap Snapshotting
Input:
All classes from application,
libraries, and VM
Iterative analysis until
fixed point is reached
Code in
Text Section
Image Heap in
Data SectionImage Heap
Writing
Output:
Native executable
16

17. Paper with Details, Examples, Benchmarks
© 2019 Oracle17
http://www.christianwimmer.at/Publications/Wimmer19a/Wimmer19a.pdf

18. Closed World Assumption
• The points-to analysis needs to see all bytecode
• Otherwise aggressive AOT optimizations are not possible
• Otherwise unused classes, methods, and fields cannot be removed
• Otherwise a class loader / bytecode interpreter is necessary at run time
•
• Dynamic parts of Java require configuration at build time
• Reflection, JNI, Proxy, resources, ...
• That’s what this talk is about
• No loading of new classes at run time
© 2019 Oracle18

19. Image Heap
• Execution at run time starts with an initial heap: the “image heap”
• Objects are allocated in the Java VM that runs the image generator
• Heap snapshotting gathers all objects that are reachable at run time
• Do things once at build time instead at every application startup
• Class initializers, initializers for static and static final fields
• Explicit code that is part of a so-called “Feature”
• Examples for objects in the image heap
• java.lang.Class objects, Enum constants
© 2019 Oracle19

20. Benefits of the Image Heap
© 2019 Oracle
Without GraalVM
Native Image
Build time
Run time
GraalVM Native Image
(default)
Build time
Run time
GraalVM Native Image:
Load configuration file
at build time
Build time
Run time
Load Classes
Load Configuration File
Run Workload
Compile Sources
Load Classes
Load Configuration File
Run Workload
Compile Sources
Load Classes
Load Configuration File
Run Workload
Compile Sources
20

21. Get VMs Ready for the Cloud and Microservices
Important evaluation metrics:
• Startup time
• Memory footprint
• Peak requests per MByte-second
© 2019 Oracle21
Bruno Borges,
Microsoft Azure Advocate

22. Currently
Startup Speed
Peak
Throughput
Memory
Footprint
Max LatencyPackaging Size
AOT JIT
© 2019 Oracle22

23. Goal
Startup Speed
Memory
Footprint
Max LatencyPackaging Size
AOT JIT
© 2019 Oracle23
Peak
Throughput

24. Java Microservice Frameworks with GraalVM Native Image Support
https://micronaut.io
https://helidon.io
https://quarkus.io
Soon also Spring support
https://github.com/spring-projects-experimental/spring-graal-native
© 2019 Oracle24

25. Startup Time
© 2019 Oracle
983 ms
1967 ms
979 ms
10.5
30 ms
23 ms
0 ms 500 ms 1000 ms 1500 ms 2000 ms
Quarkus
Micronaut
Helidon
GraalVM Native Image
JDK 8
25

26. Memory Footprint
© 2019 Oracle26
160 MByte
198 MByte
107 MByte
16 MByte
37 MByte
26 MByte
0 MByte 50 MByte 100 MByte 150 MByte 200 MByte 250 MByte
Quarkus
Micronaut
Helidon
GraalVM Native Image
JDK 8

27. Native Image vs. Java HotSpot VM
• Use GraalVM Native Image when
• Startup time matters
• Memory footprint matters
• Small to medium-sized heaps (100 MByte – a few GByte)
• All code is known ahead of time
• Use Java HotSpot VM when
• Heaps size is large
• Multiple GByte – TByte heap size
• Classes are only known at run time
© 2019 Oracle27

28. Jump Start Your Project
• How do I know quickly if my application will run as a native image?
• Disable fallback image generation
• --no-fallback
• Report unsupported features at run time
• --report-unsupported-elements-at-runtime
• Allow incomplete class path: throw linking errors at run time
• --allow-incomplete-classpath
• Trace reflection, JNI, resource, ... usage on Java HotSpot VM
• java -agentlib:native-image-agent=config-output-dir=META-INF/native-image ...
• Initialize all application classes at run time: default since GraalVM 19.0
© 2019 Oracle28

29. Reflection and JNI
• Need configuration at image build time
• Classes, methods, and fields that are reflectively visible
• Necessary to keep the metadata small and to avoid too conservative points-to analysis
•
© 2019 Oracle
class Element {
String value;
}
[
{
"name" : "com.oracle.test.Element",
"fields" : [
{ "name" : "value" }
],
"methods" : [
{ "name" : "<init>" }
]
}
]
Data class:
Element element = new Element();
element.value = "Hello World";
String json = new Gson().toJson(element);
Serialize Java object to JSON:
String json = "{"value":"Hello World"}";
Element element = new Gson().fromJson(json, Element.class);
Deserialize JSON to Java object:
Reflection configuration:
Example: Gson library to serialize / deserialize Java objects
29

30. Tracing Agent
• Trace reflection, JNI, resource usage on Java HotSpot VM
• Agent to record usage and produce configuration files for native images
• java -agentlib:native-image-agent=config-output-dir=META-INF/native-image ...
• Simplify the getting-started process
• Everything that was executed on the Java HotSpot VM also works in the native image
• Manual adjustment / addition will still be necessary
• Unless you have an excellent test suite for your application
• Fun fact: Agent is a Java Native Image
• JVMTI interface implemented using the low-level C interface of Native Image
© 2019 Oracle30

31. Blog Article with Details and Examples
© 2019 Oracle31
https://medium.com/graalvm/c3b56c486271

32. Class Initialization
• Class initialization at image build time improves application startup
• Configurable per class / package / package prefix
• --initialize-at-build-time=... --initialize-at-run-time=...
• By default, application classes are initialized at run time
• Most JDK classes are initialized at image build time
• Static analysis finds class initializers that can run at image build time
• Performance implications of class initialization at run time
• If a class is initialized at run time, no instances can be in the image heap
• Runtime checks before static method calls, static field accesses, and allocations
© 2019 Oracle32

33. Blog Article with Details and Examples
© 2019 Oracle33
https://medium.com/graalvm/c61faca461f7

34. Native Image Support in Libraries
• Configuration options should be provided by libraries
• Library and framework developers know best what their code needs
• Configuration files in META-INF/native-image are automatically picked up
• native-image.properties for command line options like class initialization options
• reflect-config.json, jni-config.json, ... for configuration files created by tracing agent
• Use subdirectories to make files composeable (inspired by Maven)
• META-INF/native-image/your.group.id/artifactId/
© 2019 Oracle34

35. Profile-Guided Optimizations (PGO)
• AOT compiled code cannot optimize itself at run time
• No dynamic “hot spot” compilation
• PGO requires relevant workloads at build time
• Optimized code runs immediately at startup, no “warmup” curve
© 2019 Oracle
native-image
--pgo-instrument
Instrumented
Binary
native-image
--pgo
Optimized
BinaryWorkloads Profiles
35

36. AOT JIT
© 2019 Oracle36

37. Tuning Native Image Footprint
-Xmx (maximum memory), default infinite
-Xmn (new generation size), default 256m
© 2019 Oracle37

38. Native Image:
Support any Platform via LLVM
• Implemented an LLVM backend for GraalVM Native Image
• In case Apple starts requiring LLVM for their apps
• Easy to port Scala to niche architectures
• Using LLVM comes with a price
• Moving GC in LLVM is still experimental
• Slow at compilation
• Slower run-time generated code
© 2019 Oracle38

39. Cross-Platform GraalVM
© 2019 Oracle
IDE
Plugin
GluonTools
JavaFX
mobile
Graal VM
Native ImageGluon
Mobile
• JavaFX extensions for mobile
• Integration with mobile functionality (e.g. GPS/camera)
• Mobile-specific connectivity
Java static
libs
39

40. Key Performance Takeaways
• Write small methods
• Local allocations are free, global data structures expensive
• Don’t hand optimize, unless you have studied the compiler graph
• Make a pull request (or at least an issue) for the GraalVM project!
• For best throughput use GraalVM JIT,
for best startup & footprint use GraalVM AOT (native images)
© 2019 Oracle40

41. Polyglot and Embeddability
© 2019 Oracle41

42. © 2019 Oracle42

43. U
U U
U
U I
I I
G
G I
I I
G
G
Node Rewriting
for Profiling Feedback
AST Interpreter
Rewritten Nodes
AST Interpreter
Uninitialized Nodes
Compilation using
Partial Evaluation
Compiled Code
Node Transitions
S
U
I
D
G
Uninitialized Integer
Generic
DoubleString
Optimization and Speculation…
© 2019 Oracle43

44. I
I I
G
G I
I I
G
G
Deoptimization
to AST Interpreter
D
I D
G
G D
I D
G
G
Node Rewriting to Update
Profiling Feedback
Recompilation using
Partial Evaluation
And Deoptimization!
© 2019 Oracle44

45. Simple Embeddability
© 2019 Oracle45

46. © 2019 Oracle
Full separation of logical and
physical data layout, enabling
virtual data structures
46

47. node modules with only JavaScript
node modules with native
extensions
node standard library
node bindings (socket, http, etc)
V8 API
thread
pool
(libeio)
event
loop
(libev)
DNS
(c-ares)
crypto
(OpenSSL)
JavaScript
C++
Java
Adapter V8 API to Graal.js via JNI
native extensions
Architecture of Node.js running via GraalVM
© 2019 Oracle
GraalVM JavaScript Engine
47

48. © 2019 Oracle48
https://kangax.github.io

49. https://www.graalvm.org/docs/reference-manual/compatibility/
© 2019 Oracle49

50. Trade-Offs
Advantages
• Java interoperability: Use any Java library or framework
• Polyglot capabilities (Python, Ruby, R)
• Run with large heaps and JVM garbage collectors
Disadvantages
• Longer start-up time
• Higher memory footprint
© 2019 Oracle50

51. Graal.js Performance (versus V8)
© 2019 Oracle51
0
0.2
0.4
0.6
0.8
1
1.2
1.4
Box2D
Crypto
Deltablue
EarleyBoyer
Gam
eboy
M
andreel
N
avierStokes
PdfJS
RayTrace
RegExp
Richards
Splay
Typescript
Zlib
Com
posite
GraalVM 19.3.0-dev V8 7.2.502

52. FastR
• GNU-R compatible R implementation
• Including the C/Fortran interface
• Built on top of the GraalVM platform
• Leverages GraalVM optimizing compiler
• Integration with GraalVM dev tools
• Zero overhead interop with other GraalVM languages
© 2019 Oracle52

53. fastRCluster package
© 2019 Oracle

54. Python Polyglot
• Use code from Java or any other GraalVM
language
• Embed into JVM languages using GraalVM
Embedder API
© 2019 Oracle54

55. Python Package Support
• Pip installer is not available, yet (c.f.: no sockets)
• We ship our own installer ginstall
• Pandas and NumPy can be installed work for a wide range of code in the latest builds
• More and more pure Python packages “just work” and compatibility is improving
© 2019 Oracle55

56. © 2019 Oracle
Enable Polyglot interoperability:
node --polyglot
56

57. Polyglot Stack Trace
© 2019 Oracle57

58. Polyglot Heap Dump
© 2019 Oracle58

59. © 2019 Oracle59

60. © 2019 Oracle60

61. 61 © 2019 Oracle
The rich ecosystem of CUDA-X libraries is now available for
GraalVM applications.
GPU kernels can be directly launched from GraalVM languages
such as R, JavaScript, Scala and other JVM-based languages.
https://github.com/NVIDIA/grcuda
© 2019 Oracle61

62. Summary
© 2019 Oracle62

63. © 2019 Oracle
Java
Scala, Groovy, Kotlin
JavaScript
Node.js
Native Image
VisualVM
Production-Ready Experimental Visionary
Ruby
R
LLVM Toolchain
Python
VSCode Plugin
GPU Integration
Webassembly
LLVM Backend
63

64. 64 © 2019 Oracle
Twitter uses GraalVM compiler in
production to run their Scala
microservices
© 2019 Oracle64

65. 65 © 2019 Oracle
• Peak performance: +10%
• Garbage collection
time: -25%
• Seamless migration
© 2019 Oracle65

66. Oracle GraalVM Enterprise Edition
• Higher performance
• Smaller footprint
• Enhanced security for native code
• Oracle Enterprise Support 7x24x365
• Support directly from the GraalVM Team
© 2019 Oracle66

67. Versioning
• Major release every 3 months named YEAR.x
• Both major and CPU releases on predictable dates
• Last major release of a year receives patches for the following year
© 2019 Oracle67

68. GraalVM Value Proposition
1. High performance for abstractions of any language
2. Low footprint ahead-of-time mode for JVM-based languages
3. Convenient language interoperability and polyglot tooling
4. Simple embeddability in native and managed programs
© 2019 Oracle68

69. Multiplicative Value-Add of GraalVM Ecosystem
• Languages GraalVM Embeddings* *
Java
JavaScript
Ruby
R
Python
C/C++, FORTRAN, …
HotSpot JVM
Oracle RDBMS
Node.js
Standalone
Spark
…
Optimizations
Tooling
Interoperability
Security
Add your own language or embedding or language-agnostic tools!
© 2019 Oracle69

70. Download and use it for your application!
• 100% production ready
• https://www.graalvm.org
• https://github.com/graalvm/
• @graalvm
© 2019 Oracle70