Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone

© 2013 IBM Corporation
Runtime Innovation:
NextGen Ninja Hacking of the JVM
Ryan Sciampacone – IBM Managed Runtime Architect
14th June 2013

© 2013 IBM Corporation2
Important Disclaimers
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR
INFORMATIONAL PURPOSES ONLY.
WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF
THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”,
WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN
A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON
HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.
ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A
GUIDE.
IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON
IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO
CHANGE BY IBM, WITHOUT NOTICE.
IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES
ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR
ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE
EFFECT OF:
- CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED
COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS

Introduction to the speaker
■  16 years experience developing and deploying Java SDKs
■  Recent work focus:
■  Managed Runtime Architecture
■  Java Virtual Machine improvements
■  Multi-tenancy technology
■  Native data access and heap density
■  Footprint and performance
■  Garbage Collection
■  Scalability and pause time reduction
■  Advanced GC technology
■  My contact information:
– Ryan_Sciampacone@ca.ibm.com

What should you get from this talk?
■  JVM proving to be a fertile ecosystem for languages
■  Plenty of opportunity to innovate in other spaces
■  Runtime is the gateway to this innovation
■  Largely ignored the last few years, but this is where the core inventions can occur

The runtime isn’t boring!

Packed Objects

Problem? What problem?
■  JNI just isn’t a great way to marshal data
■  Locality in Java can matter (e.g., JEP 142)
■  Existing native and data placement stories aren’t very good
■  In many cases, legacy systems exist – the interop is just terrible
■  So we want something that integrates well with the Java language and helps us…

What are we trying to solve?
Simple enough…
Hash
Array
Entry
(object)
(object)
Object header
Object field / data
table
key
value

Simple enough…
■  Header overhead
Hash
Array
Entry
(object)
(object)
Object header
Object field / data
table
key
value

Simple enough…
■  Pointer chasing
Hash
Array
Entry
(object)
(object)
Object header
Object field / data
table
key
value

Simple enough…
■  Pointer chasing
■  Locality
Hash
Array
Entry
(object)
(object)
Object header
Object field / data
table
key
value

int
int
Object header
Fields / data
Java Native
…
int[] d
anObject
Fighting the Java/Native interface
int
int
int
…

Ok so we have some criteria…
■  Ability to do away with headers
■  Ability to bring multiple objects close together
■  On heap / off heap seamless referencing of data
■  This actually sounds a lot like C structure types
■  Packed Objects!
struct Address {
char[4] addr;
short port;
}
struct Header {
struct Address src;
struct Address dst;
}
struct Header
port
addr Address src
Address dst
port
addr

Packed Objects: Under the covers
int y
int x
aPoint
Object header
Object field / data

int y
int x
aPoint
Object header
Object field / data
offset
target
aPackedPoint
int y
int x

Packed Objects: In Practice
int y
int x
aPoint
Object field / data
int y
int x
aPointPoint e
Point s
aLine
Object header

int y
int x
aPoint
Object header
Object field / data
int y
int x
aPointPoint e
Point s
aLine
aPackedLine
int y
int x
int y
int x
offset
target

int y
int x
aPoint
Object header
Object field / data
int y
int x
aPointPoint e
Point s
aLine
int y
int x
int y
int x
aPackedPoint s
aPackedPoint e
aPackedLine
offset
target

int y
int x
aPoint
Object header
Object field / data
int y
int x
aPointPoint e
Point s
aLine
int y
int x
int y
int x
aPackedPoint s
aPackedPoint e
@Packed
final class PackedPoint extends PackedObject {
int x;
int y;
}
@Packed
final class PackedLine extends PackedObject {
PackedPoint s;
PackedPoint e;
}
aPackedLine
offset
target

int y
int x
aPoint
Object header
Object field / data
int y
int x
aPointPoint e
Point s
aLine
int y
int x
int y
int x
aPackedLine
offset
target

int y
int x
aPoint
Object header
Object field / data
int y
int x
aPointPoint e
Point s
aLine
int y
int x
int y
int x
aPackedLine.e
aPackedLine
offset
target

int y
int x
aPoint
Object header
Object field / data
int y
int x
aPointPoint e
Point s
aLine
int y
int x
int y
int x
aPackedLine.e
offset
target
aPackedPoint
aPackedLine
offset
target

Packed Objects: In Practice with Native Access
int y
int x
Object header
Struct field / data
Java Native
struct Point {
int x;
int y;
}

int y
int x
Object header
Struct field / data
offset
target
aPackedPoint
Java Native
struct Point {
int x;
int y;
}
@Packed
final class PackedPoint
extends PackedObject {
int x;
int y;
}

int y
int x
Object header
Struct field / data
offset
target
aPackedPoint
Java Native
struct Point {
int x;
int y;
}
Ø
@Packed
final class PackedPoint
extends PackedObject {
int x;
int y;
}

Lets Build Something in C!
■  Nested substructures
■  Compact representation
■  Alignment aspects
struct Address {
char[4] addr;
short port;
}
struct Header {
struct Address src;
struct Address dst;
}
struct Header
port
addr Address src
Address dst
port
addr

Let’s Build the Same “Something” in Java!
■  Headers
■  No locality
■  Alignment
class Address {
byte[] addr;
short port;
}
class Header {
Address src;
Address dst;
}
Address
port
addr
addr
Header
Address dst
Address src
Address
port
addr
addr
byte[]
byte[]

What does the Java code look like under the covers?
■  From a code point of view, this isn’t terrible…
Bytecodes:
aload1
getfield Header.dest LAddress;
getfield Address.addr [B
iconst0
baload
bipush 192
ificmpeq ...
JIT (32 bit):
mov EBX, dword ptr -4[ECX] // load temp1
mov EBX, dword ptr 8[EBX] // load dest
mov EBX, dword ptr 4[EBX] // load addr
movsx EDI, byte ptr 8[EBX] // array[0]
cmp EDI, 192

What if we did this with Packed Objects?
■  The Java code is pretty clean… and a pretty good result!
@Packed
final class Address extends PackedObject {
PackedByte[[4]] addr;
short port;
}
@Packed
final class PacketHeader extends PackedObject {
Address src;
Address dest;
}
port
addr Address src
Address dst
port
addr
offset
target
aPackedHeader

Ok, what about the code under the covers?
■  Bytecodes don’t change… JIT code is pretty good too!
JIT (32 bit):
mov EBX, dword ptr -4[ECX] // load temp1
mov EAX, dword ptr 4[EBX] // load target
mov EDX, dword ptr 8[EBX] // load offset
lea EBX, dword ptr [EAX + EDX]
movsx EDI, byte ptr 8[EBX] // array[0]
cmp EDI, 192
Bytecodes:
aload1
getfield PackedHeader.dest LAddress;
getfield Address.addr [B
iconst0
baload
bipush 192
ificmpeq ...

What about native access?
Java Native
…
anObject
How do we implement this normally?
port
addr
port
addr

JNI implementation
■  Usual “stash pointers in long types” tricks
■  JNI costs tend to be high

DirectByteBuffer implementation
■  No extra JNI to write (this is good)
■  Keeping your indices straight is never fun

Unsafe implementation
■  You shouldn’t be here
■  Still playing the indices game

PackedObject answer
■  Looks like natural Java code
■  Foregoes JNI
■  Same type capable of on-heap representation
port
addr
port
addr
offset
target
aPackedHeader
Ø

Active work and next steps
■  Experimenting with this now
■  Yes, there are security aspects to be aware of here
■  This is potentially part of a larger look at Java / Platform interop
■  Not specifically viewed as a cure to GC problems
■  This forms the basis for many other solutions to existing problems…

Multitenancy

Just what do you mean by “multitenancy”?
With a multitenant architecture, a software application is designed
to virtually partition its data and configuration, and each client
organization works with a customized virtual application
instance.
■  Working Definition
– A single instance of a software application that serves multiple customers
à Each customer is a tenant.
– Tenants can customize some parts of the application (look and feel) but not the code.
– Infrastructure usually opaque
à opportunity for provider
Why? Cost Savings: As compared to single-tenant deployment model

S5. Shared
Application
JDK Support for Spectrum of Sharing / Multitenancy (Level 1-5)
S1. No Sharing
Infrastructure
Middleware
Application
Tenant
OS
Middleware
Application
Tenant
OS
Infrastructure
Middleware
Application
Tenant
OS
Middleware
Application
Tenant
OS
Infrastructure
Data Center floorData Center floor
Middleware
Application
Tenant
Middleware
Application
Tenant
Infrastructure
Data Center floor
OS
Application
Tenant
Application
Tenant
Infrastructure
Data Center floor
OS
Middleware
Application
data
Tenant Tenant
Infrastructure
Data Center floor
OS
Middleware
Application
S2. Shared
Hardware
S3. Shared
Operating System
S4. Shared
Middleware
Application
data
Application
data
Application
data
Application
data
Application
data
Application
data
Application
data
Application
data
Application
data
Sharing servers
storage, networks in a
data center
Hypervisors (e.g. KVM,
VMWare) are used to
virtualize the hardware
Multiple applications
sharing the same
middleware
Sharing the same
application
Multiple copies of
middleware in a single
operating system
Application
Changes
Application
Changes
? Application
Changes

= LApplication
Changes

Hardware Virtualization
■  Hypervisors run multiple applications side-by-side safely
■  Advantages
– Capture idle CPU cycles
– Automatic de-duplication (RAM)
– Ability to meter and shift resource toward demand
– No need to change tenant applications
Hypervisor
Hardware
tenant tenant tenant tenant

Hardware Virtualization
■  Hypervisors JVMs can run multiple applications side-by-side safely
■  Advantages
– Capture idle CPU cycles
– Automatic de-duplication (ability to share Java artifacts)
– Ability to meter and shift resource toward demand
– No need to change tenant applications
Hypervsisor
Hardware
tenant tenant tenant tenant
Java VM
Operating System

Multitenancy: Low (or no) barrier to entry
■  Multitenancy is all about reducing duplication by transparently sharing a JVM
– 1 GC, 1 JIT, shared heap objects
– plus: JVM-enforced resource constraints to meter and limit consumption
■  Ergonomics: Opt-in to multitenancy with a single flag: -Xmt (multitenancy)
– no application changes required
javad
Tenant1
Tenant2
One copy of common code + data
lives in the javad process.

JVM: Separating State
■  Static variables are a problem for sharing
■  Consider use of System.out in code we want to share below

getstatic does 2 things
1.  Triggers class initialization on first contact
–  Danger: Each ‘tenant’ needs to do this
2.  Resolves a name (out) to a storage location and reads from it
–  Danger: Each tenant needs dedicated storage

JVM: Standard Static Fields
getstatic <cpIndex>
0: if constantPool[cpIndex] is resolved then {
1: fetch field address from constantPool
2: read field value from class
3: return field value
4: } else {
5: initialize class
6: determine address of field
7: store field address into constantPool
8: goto 1
9: }

JVM: Isolated Static Fields
getstatic <cpIndex>
0: if constantPool[cpIndex] is resolved then {
1: fetch field address index and class from constantPool
2: fetch tenant->data[class->initIndex]
3: if (!initialized) {
4: initialize class;
5: }
6: read field value from tenant->data[index]
7: return field value
8: } else {
9: initialize class
10: determine address index of field
11: store field address index into constantPool
12: goto 1
13: }

■  There are other initialization triggers too
– putstatic
– invokestatic
– instanceof
– new
■  Some of these warrant special handling as they are heavily used
(invokestatic) and initialization checks are expensive.
– Separate stack frame build from opcode

More that just JVM State…
■  Throttling of resources
– Threads, GC, sockets, files (IO in general), native memory
■  Past and existing examples do exist!
– Commercial / In house custom solutions
– JSR 181 Isolates / 284 Resource Management
■  Security is of course huge

Other Thoughts – Native Libraries and shared state
■  Use separate processes to manage different state
■  Each process now holds the context
■  Challenges: Latency
Tenant1
Tenant2
JVM
Proxy
Library
Shared
Library
Shared
Library
Tenant1
Tenant2
JVM
Shared
Library
Shared
State!
■  Native libraries contain state that may not be shareable across tenants

Heap Memory
(not Garbage Collection!)

Scaling Garbage Collection
■  Existing Garbage Collection (GC) technology defers expensive costs
Heap
Local
Collection
Global
Collection
Heap
■  Global approach introduces linear scaling problem (100ms @ 1GB → 10s @ 100GB)
■  Goal: Scale using result-based, incremental (through partitioned heap) garbage collection
■  Target appropriate data sets based on environment factors
■  Key: Region Based Garbage Collection approach

Challenge: Segregate Memory by Differentiators
Heap
Varying memory
characteristics and
requirements per
application
JVM must
dynamically
recognize and adapt
given platform
resources
 IaaS optimization + sharing (consolidate)
 SSD Exploitation (memory efficiency)
 Multi-core scalability (scale up)
 Very Large Heap GC (low pause times)
 Soft Real-Time
■  Objects with common properties
•  Locality
•  Usage frequency
•  Lifetime
•  Levels of “Read-only”ness
■  Optimize: Memory characteristics
•  Page sharing status
•  Memory speed aware
•  Flash / SSD exploitation
■  “Results based” incremental
operations
•  Productive GC every cycle
•  Localized garbage collect
Custom
 Placement improves cache performance
 Allocation efficiency (thoughtput)
 Reduced size working set (paging)

Memory Hierarchy and Performance
■  Increased core and HW thread counts are not silver bullets to scaling in a
memory intensive environment such as Java
■  Pressure from increased
cores / HW threads
■  Cache hierarchy does not
scale
Cache
Hierarchy
BUS
Vs.
Logical Processors
Memory
MemoryMemory
Memory
CPU CPU
CPUCPU
■  Memory speed varies based on
execution context
■  Magnitude of variance related
to CPU socket count
■  Uneven application performance over time
•  Data access can be fast or slow (or both)
•  Behavior can be variable run to run due to data placement
•  Threads scheduling can be unpredictable
•  Same deployments on different hardware cause unpredictable performance shifts

Let’s look at transferring data
Heap
A
B
C

Heap
A
B
C
Remote
Transfer

PackedObjects could help…
Heap
A
Remote
Transfer
C
B

Heap
A
B
C
Remote
Transfer
A
Packed
C
B

Heap
A
B
C
Remote
Transfer
A
Packed
B
C
A
Packed
C
B

Making the data transfer easier…
Heap
Remote
Transfer

Heap
Remote
Transfer
Specialized Heap Area

Heap
B
Remote
Transfer
A
B
C
A
C

Making the data transfer seamless
Heap
Remote
Transfer

B
C
A
B
C
A
Heap
Remote
Transfer
B
C
A

B
C
A
B
C
A
Heap
Remote
Transfer
B
C
A
Packed
offset
target
Packed
offset
target

Tip of the Iceberg
■  Just scratching the surface of what is possible
■  GC technology has an effect on what is possible
– But this should be invisible to the application and seamless in the language
■  Innovation in the heap space is not limited to GC

Virtualization

Finding your way in The Cloud
■  Constant pressure to reduce cost and deliver more services for a given cost
■  Maximize the available resources (Hardware)
■  CPU and Memory are typically the biggest
– Other critical ones as well, including I/O
■  Plenty of solutions
– Cloud and Multitenancy just to name a few
■  And so by and large we’re in a Virtualization scenario

The Layers of Liars
■  Your application could end up querying 3 different layers to get a (hopefully) correct picture
Hypervisor
Hardware
OS
API! availableProcessors() [no accuracy / understanding]
Thinks it knows but doesn’t (being lied to)
Knows and controls, but can change its mind
App

The Layers of Liars
Hypervisor
Hardware
App
OS
App
Image source (Wikipedia Commons)
CPU: https://en.wikipedia.org/wiki/Central_processing_unit
DRAM: http://en.wikipedia.org/wiki/Dynamic_random-access_memory
RAM
CPU

The Layers of Liars
Hypervisor
Hardware
App
OS
App
RAM
CPU

The Layers of Liars
Hypervisor
Hardware
App
OS
App App
OS
RAM
CPU

The Layers of Liars
■  You don’t even have a constant picture of the universe
Hypervisor
Hardware
App
OS
App App
OS
RAM
CPU

Knowing what you don’t know
(What does this even mean anymore?)

Interfaces… more importantly behavior
■  Provide interfaces to the “Layers of liars” to enable middleware “smarts”
■  Runtime reads and reacts (in co-operation with application / other runtimes)
Hypervisor
Hardware
App
OS
Hypervisor
Hardware
App
OS
App App
OS
Adjust –Xmx
Adjust –Xgcthreads
Adjust thread pool counts
…

…And after all that

Tenant
Nimble Reactions to External Events
Hypervisor
Hardware
OS
Application
Tenant

Tenant
Hypervisor
Hardware
OS
Application
Tenant

TenantTenant
Hypervisor
Hardware
OS
ApplicationApp
OS
Tenant

TenantTenant Tenant Tenant
Hypervisor
Hardware
OS
ApplicationApp
OS
Tenant
Hypervisor
Hardware
OS
Application

TenantTenant Tenant Tenant
Hypervisor
Hardware
OS
ApplicationApp
OS
Tenant
Hypervisor
Hardware
OS
Application
Tenant
RDMA + Packed Objects

Questions?

References
■  Get Products and Technologies:
– IBM Java Runtimes and SDKs:
•  https://www.ibm.com/developerworks/java/jdk/
– IBM Monitoring and Diagnostic Tools for Java:
•  https://www.ibm.com/developerworks/java/jdk/tools/
■  Learn:
– IBM Java InfoCenter:
•  http://publib.boulder.ibm.com/infocenter/java7sdk/v7r0/index.jsp
■  Discuss:
– IBM Java Runtimes and SDKs Forum:
•  http://www.ibm.com/developerworks/forums/forum.jspa?forumID=367&start=0

Copyright and Trademarks
© IBM Corporation 2012. All Rights Reserved.
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of
International Business Machines Corp., and registered in many jurisdictions
worldwide.
Other product and service names might be trademarks of IBM or other companies.
A current list of IBM trademarks is available on the Web – see the IBM “Copyright
and trademark information” page at URL: www.ibm.com/legal/copytrade.shtml

Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone

Recommandé

Recommandé

Contenu connexe

Similaire à Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone

Similaire à Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone (20)

Plus de ZeroTurnaround

Plus de ZeroTurnaround (20)

Dernier

Dernier (20)

Runtime Innovation - Nextgen Ninja Hacking of the JVM, by Ryan Sciampacone