Put Your Java Apps to Sleep and Improve Efficiency

Put
your
Java
Apps
to
Sleep?

Find
out
How…

John
Ma(hew
Holt
|
JAX
London
2013

!

© Copyright Waratek 2013

John Matthew Holt!
•  CTO
and
co-‐Founder
of
Waratek

•  Been
doing
JVM
engineering
for
10
years

•  Invented
“Java
VirtualizaCon”
to
transform

how
JVMs
operate
in
virtualized
and
cloud-‐
compuCng
environments

•  Lead
the
design
of
the
ﬁrst
Type-‐1
VMM
for
a

JVM


What will we cover?!
•  First
some
background…

– Why
virtualize
INSIDE
the
JVM?

– Think
“VMware
for
Java”

•  PuSng
you
Java
Apps
to
Sleep…

– Why
are
Java
apps
Cred?

(or
restated:
why
should
they
be
slept?)

– Let
your
apps
nap:
“wake-‐on-‐LAN
for
JVCs”


First some background…!

•  Why
virtualize
INSIDE
the
JVM?


Because…!
•  Because
JVMs
make
bad
bedfellows
with
HW

hypervisors
(HV)

•  A
JVM
thwarts
HW
hypervisor
features
like:

–  The
GC
in
a
JVM
thwarts
memory-‐overcommit
in
a
HW

HV

–  The
GC’s
heap
memory,
and
the
JIT’s
permgen/
codecache,
thwarts
‘transparent
page

sharing’
(memory
de-‐dup)
in
a
HW
HV

–  Dedicated
GC
and
JIT
for
EVERY
Java
applicaCon
wastes

lots
and
lots
of
CPU
Cme
and
memory
capacity
doing

the
same
thing
over,
and
over,
and
over,
and…


Today’s JVMs are OLD!
•  Were
designed
~20
years
ago,
and
haven’t

changed
design
since
then

•  JVM
design
PREDATES
the
major
2
trends
of

last
decade:
virtualizaCon
and
cloud-‐compuCng

•  Only
2
innovaCons
in
JVMs
in
20
years:

– Beder
JIT
compilers

– Concurrent
GCs

– ….all
blindly
focused
on
single-‐tenant
performance

at
the
expense
of
mulC-‐tenant
eﬃciency


Server Java has problems!
•  JVMs
are
grossly
ineﬃcient
users
of
CPU
and
memory

–  A
JVM
assumes
it
can
use
all
available
CPU
and
memory
capacity

for
itself
and
its
single
applicaCon,
eg:

–  1
JBoss
can
consume
up
to
500MB
RSS
with
no
apps

–  1
Tomcat
can
consume
up
to
250MB
RSS
with
no
webapps

–  Running
mulCple
JVMs
on
a
server
rapidly
exhausts
physically

memory
while
CPU
uClisaCon
averages
5%

•  >100
diﬀerent
JVM
versions
makes
compliance/support
a

NIGHTMARE

•  Security
vulnerabiliCes
and
compromises
are
EVERYWHERE

inside
Java


VMWare does not virtualize  
Java well!
•  Realworld
stats
using
VMware:

–  >80%
of
Server
JVMs
have
<=2GB
of
heap
memory

•  Yet
actual
applicaCon
heap
uClisaCon
averages
<20%

–  A
2GB
JVM
heap
needs
~3GB
of
OS
RSS
memory,
which

in
turn
needs
~4GB
of
VMWare
tenant
memory

•  …reducing
actual
applicaCon
memory
uClisaCon
to
<10%

–  CPU
uClisaCon
running
Java
on
Vmware
averages
<7%

•  Yet
physical
memory
is
exhausted
because
of
running

mulCple
JVMs


Java is a compliance nightmare!
•  Realworld
stats:

–  93%
of
organisaCons
are
running
versions
of
Java
at

least
5
years
old

–  The
average
organisaCon
has
51
disCnct
versions
of

Java
installed

–  <1%
of
installed
Java
is
upgraded
to
the
latest
version

•  Recurring
enterprise
conversaCon:

–  QuesCon:
“what
version
of
Java
do
you
run?”

–  Answer:
“every
version
except
the
latest
version!”


Java is a security nightmare!
•  Realworld
stats:

–  In
last
12
months
alone,
>250
CVE
alerts
issued
by
US-‐
CERT/NIST
for
Java

–  The
most
prevalent
version
of
Java
has
96
“perfect
10”

vulnerabiliCes

–  “There
are
virtually
no
modern
versions
of
Java
without

any
known
severe
vulnerabiliCes”
(source:
Bit9)

•  CriCcal
Security
Patch
Updates
are
released
so

oven
that
developers
and
admins
can’t
keep
up


Virtualization is the answer…!
•  …but
HIGHER
up
the
Server
Stack

•  VirtualizaCon
is
creeping
up
the
Server
Stack

already

–  x86
HW
got
virtualized
in
1999
(VMware
et
al)

–  x86
OS
got
virtualized
in
2006
(OpenVZ,
LXC,
et
al)

–  …now
it’s
Java’s
turn
in
2013

•  Virtualizing
Java
means
virtualizing
at
the
TOP
of

the
Server
Stack,
whereas
VMware
virtualizes
at

the
BOTTOM


Cost-per-tentant to zero !
•  The
higher
up
virtualizaCon
goes
up
the
Server

Stack,
the
cheaper
a
“tenant”
becomes

•  Virtualizing
INSIDE
the
JVM
gets
close
to
“zero-‐
cost
tenants”:

– Helloworld
in
<1MB
of
tenant
memory

– Examples.war
(Tomcat
demo
webapps)
in
<5MB
of

tenant
memory

– …Java
becomes
CHEAPER
than
python/ruby/perl

when
the
JVM
itself
gets
virtualized!


Virtualization == Security!
•  Virtualizing
the
JVM
makes
Server
Java
secure
and

compliant

–  Virtualized
Java
Apps
run
as
“tenants”
in
a
shared
JVM

–  Tenants
run
in
Java
Virtual
Container
(JVC)
“jails”

–  Tenants
are
isolated
and
“locked-‐down”

–  Tenants
can
run
any
Java
version

•  Doesn’t
mader
what
“Java
version”
a
tenant
uses,
it
gets
the
latest

JVM

•  Means
a
“Java
version”
becomes
just
a
tenant
API
for
the

convenience
of
developers

•  Admins
control
the
REAL
version
at
the
JVM
level
so
that
tenants

(developers)
can
stop
caring
about
“versions”
altogether


First some background…!

•  Think
“VMWare
for
Java”


A hypervisor INSIDE a JVM!
•  Waratek
has
wriden
a
type-‐1
VMM
that
runs
INSIDE
the

Oracle
HotSpot
and
OpenJDK
JVMs

–  >300,000
lines
of
Java
code,
<5K
lines
of
C-‐code

–  Can
easily
run
on
any
OS/CPU
pair
(only
Linux/x86
supported
so

far)

–  Does
EVERYTHING
VMware
does,
but
INSIDE
the
JVM

• 
• 
• 
• 
• 
• 

Virtual
FS

Virtual
networks/IPs/NICs

CPU
pinning
and
quotas

Memory
overcommit,
elasCc
memory,
memory
quotas

I/O
QoS,
rate-‐limiCng

Etc…

–  …all
with
ZERO
code
change
to
Java
applicaCons


A real VMM inside the JVM!


VMWare for Java!
Java Code
JVC
Java App
java.* APIs
SecurityManager
Waratek Java
Hypervisor
Process

C/C++ Code

JNI
Libs

•  Waratek
virtualizes
Java
exactly

like
VMware
virtualizes
HW

–  Waratek
inserts
a
“Java

Hypervisor”
between
the
JVM
and

the
Java
App
+
java.*
APIs
+

SecurityManager

–  The
Java
app
PLUS
java.*
APIs
and

SecurityManager
run
inside
a
Java

Virtual
Container
(JVC)
“jail”

–  For
a
Java
App
to
go
outside
of
its

JVC
“jail”
it
has
to
“trap”
to
the

Java
Hypervisor

Waratek
JVM

VMWare for Java!
Java Code
JVC
Java App
java.* APIs
SecurityManager
Waratek Java
Hypervisor
Process

C/C++ Code

JNI
Libs

Waratek
JVM

•  Every
incoming
and
outgoing
access

by
a
JVC
is
intercepted
and
checked
by

the
Java
Hypervisor,
e.g:

– 
– 
– 
– 
– 
– 
– 
– 
– 
– 
– 
– 
– 
– 

Class
loading
intercepted
and
checked

JNI
invocaCon
intercepted

Parameter
and
return
value
checks

File
access
(delete/copy/move)
checks

Network
operaCon
checks

IO
rate-‐limiCng
and
flow-‐control

CPU
Cme
control

Memory
consumpCon
control

Buffer
overflow
checks

Null
pointer
checks

Type
safety
checks

OS
access
checks

JNI
Libraries
isolated

…


Slots into existing VM tools!
•  You
can
manage
the
Java
hypervisor
exactly
like
you
manage

exisCng
HW
hypervisors:

– 
– 
– 
– 

SSH
CLI

RedHat
LibVirt/KVM
integraCon
(beta)

VMWare
vCAC
integraCon
shortly

OpenStack
integraCon
shortly

•  Plus
other
management
interfaces

– 
– 
– 
– 
– 

HTTP
REST
API

Local
and
remote
JMX
APIs

/proc/PID/*
pseudo-‐ﬁlesystem
for
JVM
and
each
JVC

Vmware
CloudFoundry
integraCon
(beta)

RedHat
OpenShiv
integraCon
soon


A HelloWorld JVC in LibVirt XML!


JVCs make Java secure!
Java Code
JVC
Java App
java.* APIs
SecurityManager
Waratek Java
Hypervisor
Process

C/C++ Code

JNI
Libs

Waratek
JVM

•  If
a
Java
App
inside
a
JVC

compromises
either/both
of

java.*
APIs
or
SecurityManager,

then
only
that
JVC
is
effected

and
NOT
the
Java
Hypervisor
or

JVM
or
other
JVCs

•  JNI
Libraries
are
isolatable
in

their
own
OS
process

•  Thus
a
compromised
JVC

cannot:

– 
– 
– 
– 
– 
– 
– 

Insert
malicious
code
outside
its
JVC

Delete/move/copy
files
outside
its
VROOT

Trigger
buffer
overflows

Do
null-‐pointer
segmentaCon
faults

Perform
unsafe
heap
memory
access

Fork
OS
processes

etc…


“Come in, we’re open”!
•  The
Java
Hypervisor
has
an
open
API
for
unlimited

customizaCon
and
extensibility

•  Anyone
can
write
their
own
PURE-‐JAVA
plugins
to

the
Java
hypervisor
to
customize
ANYTHING
about

JVCs,
eg:

–  Make
a
DropBox.com
account
the
virtual
ﬁlesystem
for

a
JVC

–  Create
a
virtual
LAN
between
distributed
JVCs

anywhere
in
the
world

–  Clone
live
network
I/O
of
an
acCve
JVC
to
a
test
JVC
for

side-‐by-‐side
applicaCon
tesCng


Example hypervisor plugin!
•  Waratek
wrote
ElasCcat
as
one
example
of
a
Java

Hypervisor
plugin

•  ElasCcat
is
open
source,
is
<1000
lines
of
Java
code,
and

loads
into
the
Java
hypervisor
at
startup

•  ElasCcat
virtualizes
Apache
Tomcat
so
it
becomes
the

world’s
ﬁrst
mulC-‐tenant
JavaEE
container:

–  Tomcat
runs
in
one
JVC
and
each
webapp
runs
in
it’s
own

dedicated
JVC

–  If
one
webapp
crashes
it
won’t
crash
any
other
JVC

–  …all
this
achieved
with
ZERO
code
change
to
Tomcat
or

webapps,
and
without
them
knowing
they’ve
been

virtualized!


Putting Java to jSleep…!

•  Why
are
Java
apps
Cred?

(or:
why
should
they
be
slept?)


Java Apps are tired!!
•  Realworld
stats:

– >50%
of
PaaS
apps
are
“dead
bone
idle”
at
any

given
Cme

– The
average
Dev/Test
environment
runs
at
<10%

sustained
uClizaCon

– Lots
and
lots
of
enterprise
Java
apps
are
only
used

periodically,
e.g.
end-‐of-‐week,
end-‐of-‐month…

•  But
these
apps
have
SLAs
and
can’t
be
shut-‐oﬀ!


Fact from the field:!
•  So
frustrated
was
one
Fortune500
admin
that
he

shut-‐off
a
bunch
of
Java
apps
that
had
been
idle
for

2
weeks
(no
TCP
traffic)
and
waited
to
see
what

would
happen…
the
results
were:

–  …a
small
number
of
app-‐owners
noCced
immediately

(1-‐2
days)

–  …the
largest
number
of
app-‐owners
noCced
within
1-‐2

weeks

–  …and
a
small
number
of
app-‐owners
NEVER
noCced
at

all!


Wouldn’t it be great if…!
•  …you
could
“compress”
an
idle
app
and
re-‐
awaken
it
instantly
WITHOUT
THE
APP

KNOWING?

•  …and
without
the
END-‐POINTS
knowing?

•  …and
without
ever
RESTARTING
the
app
later?


Actually it exists…almost!!
•  The
technology
for
this
nearly
already
exists:

–  It’s
called
“wake-‐on-‐LAN”
(WoL)

•  …but
there’s
a
problem:
WoL
only
works
at
the

HARDWARE
level,
so
is
of
NO
USE
in
mulCtenant

virtualized
cloud-‐compuCng
environments

•  Think
about
it:

–  As
long
as
ONE
tenant
on
a
server
is
acCve
then
the
server

HAS
to
stay
alive
and
WoL
CANNOT
be
used!


Putting Java to jSleep…!

•  Let
your
apps
nap:
wake-‐on-‐LAN

for
JVCs


“Wake-on-LAN for JVCs”!
•  What’s
needed
is
a
NEW
kind
of
WoL
that

“naps”
single
idle
tenants
on
a
mulCtenant

server…

•  …and
reclaims
the
memory/CPU/IO
quotas

that
were
provisioned
for
the
now-‐napped
app,

to
be
used
to
run
other
new
apps

•  This
is
what
Waratek
jSleep
does
for
Java
apps!


jSleep in a Slide!
•  Waratek
jSleep
works
inside
the
Java
Hypervisor
to

do
four
things:

–  1)
monitor
the
acCvity
of
individual
apps
(JVCs)

–  2)
put
idle
JVCs
to
sleep
aver
deﬁned
inacCvity
thresholds

and
reclaim
their
CPU/mem/IO
quotas

–  3)
monitor
the
I/O
connecCons
(TCP/UDP
Sockets,

ServerSockets,
Files,
etc)
of
a
sleeping
JVC

–  4)
and
awaken
a
sleeping
JVC
when
I/O
acCvity
is

detected,
reinstaCng
reclaimed
quotas

•  …it’s
wake-‐on-‐LAN
for
JVCs!


jSleep is Very Powerful!
•  Lots
of
“knobs
and
dials”
make
it
possible
to
define

sophisCcated
sleep/awake
profiles
that
respond
to:

–  CPU
acCvity

–  Network/IO
acCvity

–  IO
Cmeouts

–  Wall-‐clock
Cme

–  …or
just
about
anything
else

•  THERE’S
ALSO
a
built-‐in
network
profile
that
allows

for
immediate
configuraCon
via
the
JIRSH
CLI


Simple jSleep Example!
•  Set
a
jSleep
policy
based
on
network
traffic
of
defined

sockets/ports,
eg:

–  jirsh # sleepinfo 1 –-set
network=120000,include_local=:8080,10.1.0.8:4444

–  Above
command:

•  Suspends
JVC
1
aver
the
network
interface
is
idle
120s

•  The
include_local param
is
a
series
of
host:port
combinaCons

that
specify
what
traffic
of
remote
hosts
and/or
local
ports
is
allowed

to
wake
the
JVC,
so
in
above:

–  Traffic
from
any
remote
host
to
local
port
8080
will
wake
the
JVC

–  Traffic
from
remote
host
10.1.0.8
to
local
port
4444
will
wake

the
JVC


jSleep API in 60 Seconds!
•  2
packages,
8
classes:

–  cloudvm.jsleep.event.*
• 
• 
• 
• 

BytesReceivedEvent
CpuEvent
NetworkIdleEvent
TimeoutEvent

–  cloudvm.jsleep.profile.*
• 
• 
• 
• 

JSleepProfile
JSleepProfile.MemoryType
JSleepProfileState
NetworkProfile


jSleep API in 60 Seconds!
•  Main
class:
cloudvm.jsleep.profile.JSleepProfile
–  Every
JVC
can
have
one
of
these

–  Defines
the
two
CORE
methods:

•  checkSuspend( VirtualContainerMXBean )
•  checkResume( VirtualContainerMXBean )

–  Is
an
abstract
class
so
anyone
can
write
their
own
custom

sleep/resume
profile

–  Loaded
and
configured
using
JMX
(local
or
remote)


Custom jSleep API Example!
•  Here’s
a
custom
jSleep
proﬁle…

public
boolean
checkSuspend(VirtualContainerMXBean
paramVirtualContainerMXBean)
{

Date
currentDate
=
new
Date();

if(currentDate.aver(sleepDate)
&&
currentDate.before(resumeDate))
{

return
true;

}
else
{

return
false;

}

}

public
boolean
checkResume(VirtualContainerMXBean
paramVirtualContainerMXBean)
{

Date
currentDate
=
new
Date();

if(currentDate.before(sleepDate)
||
currentDate.aver(resumeDate))
{

return
true;

}
else
{

return
false;

}

}


What happens when…!
•  …a
JVC
sleeps?:

–  The
threads
of
the
JVC
are
suspended
on
the
OS
kernel

(but
NOT
destroyed)

–  The
JVC’s
dead
objects
are
asynchronously
reclaimed
by

the
GC

–  If
the
JSleepProfile
was
configured
with

MemoryType.VARIABLE,
the
JVCs
unused
memory
quota

will
be
reclaimed
for
use
by
other
JVCs

–  All
open
file
descriptors
will
be
kept
open

–  …all
this
in
<1
second


What happens when…!
•  …a
JVC
wakes
up?:

– The
threads
of
the
JVC
are
resumed
on
the
OS

kernel

– If
the
JSleepProfile
was
configured
with

MemoryType.VARIABLE,
the
JVM
will
reinstate

the
prior
memory
quota
of
the
JVC

– Any
file
descriptors
with
received
but
unread
data

will
now
be
“readable”
by
the
applicaCon

– …all
this
in
<1
second


Why you should care…!
•  Dev/Test,
PaaS,
and
middleware
environments
are

characterized
by
LOTS
of
“applicaCon
idleness”

•  Idle
apps
today
are
treated
NO
DIFFERENTLY
than
acCve,

busy
apps

•  Years
ago
H/W
wake-‐on-‐LAN
was
an
eﬀecCve
technology
for

powering
down
idle
servers…
but
it
no
longer
works
in

mulCtenant
cloud
infrastructure

•  jSleep
is
“wake-‐on-‐LAN
for
JVCs”
and
can
increase
the

number
of
Java
apps-‐per-‐server
by
up
to
10x

–  …this
means
a
reducCon
in
server
sprawl
and
OpEx
up
to
90%


Summary!

•  Recap
and
links…


Biggest change to Java in yrs!
•  A
hypervisor
inside
the
JVM
transforms
Java

with:

– Ultra-‐liteweight
JVCs
for
near
zero-‐cost
tenants

– JVC
“jails”
for
Java
security
and
isolaCon

– Mixed-‐version
JVCs
for
version
consolidaCon
and

legacy
compliance

– jSleep
(“Wake-‐On-‐LAN
for
JVCs”)
for
near-‐zero

footprint
of
idle
tenants


Links!
•  General
informaCon
about
the
Waratek
JVM

– hdp://www.waratek.com

•  jSleep,
JVI
(Java
VirtualizaCon
Interface),
and

JMX
APIs

– hdp://javadoc.waratek.com

•  ElasCcat
source
code

– hdps://github.com/waratek/elasCcat-‐driver


Put Your Java Apps to Sleep and Improve Efficiency

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Put Your Java Apps to Sleep and Improve Efficiency

Similaire à Put Your Java Apps to Sleep and Improve Efficiency (20)

Plus de jaxLondonConference

Plus de jaxLondonConference (18)

Dernier

Dernier (20)

Put Your Java Apps to Sleep and Improve Efficiency