In this presentation, Andrew Spyker and I present our experience with adopting Netflix OSS, both from a deep runtime perspective for various applications and services as well as managing deployed services for scalability and failover.

1. 1
IBM Confidential
Streaming Movies brings you Streamlined Applications --
How Adopting Netflix Libraries can Improve Your Application
or Service!
March, 2014
Andrew Spyker, STSM Michael Elder, STSM
aspyker@us.ibm.com mdelder@us.ibm.com
@aspyker @mdelder

2. 2
2012
2013
2014
Beyond
SPECjEnterprise
AcmeAir
Cloud/Mobile
Sample/Benchmark
born
Sample
applicaBon
cloud
prize
work
AcmeAir
Run
On
IBM
Cloud
at
“Web
Scale”
Portability
cloud
prize
work
Scalable
Services
Fabric
internally
for
IBM
Services
Landscaper
&
JazzHub
adopBon
of
NOSS
libraries
Historical
Context
SoSLayer
and
BlueMix
services

3. Ne0lixOSS
• “Technical
indigesBon
as
a
service”
– Adrian
CockcroS
• neWlix.github.io
– 40+
OSS
projects
– Expanding
every
day
• Focusing
more
on
interacBve
mid-­‐Ber
server
technology
today
…

4. What
is
service
opera7onal
excellence?
¡ OperaBng
a
24/7
producBon
public
cloud
service
with
paying
customers
means
…
¡ High
availability
¡ No
SPOF’s,
all
components
at
least
triple
clustered
¡ AutomaBc
Recovery
¡ ParBal
failure
should
be
recovered
by
system
¡ ConBnuous
Delivery
¡ Changes
delivered
frequently
with
zero
downBme
¡ ElasBc
Scalability
¡ SoluBon
can
scale
out
easily
¡ OperaBonal
Visibility
¡ Operators
have
live
view
of
contextualized
state
of
system

5. Micro
service
ImplementaBon
Call
“Auth
Service”
Ribbon
REST
client
with
Eureka
Web
App
Front
End
(REST
services)
App
Service
(auth-­‐service)
Execute
auth-­‐service
call
Hystrix
Eureka
Server(s)
Eureka
Server(s)
Eureka
Server(s)
Karyon
&
Archaius
Fallback
ImplementaBon
Implementa7on
Detail
Benefits
Decompose
into
micro
services
• Key
user
path
always
available
• Failure
does
not
propagate
across
service
boundaries
Karyon
/w
automaBc
Eureka
registraBon
using
Archaius
for
property
management
• New
instances
are
quickly
found,
failing
individual
instances
disappear
• Per
deployment
environment
composite
property
sources
provide
dynamic
behavior
at
runBme
Ribbon
client
with
Eureka
awareness
• Load
balances
&
retries
across
instances
with
“smarts”
• Handles
temporal
instance
failure
Hystrix
as
dependency
circuit
breaker
• Allows
for
fast
failure
• Provides
graceful
cross
service
degradaBon/recovery
Highly
Available
Service
Run7me
Recipe

6. Region
(Dallas)
DAL01
Datacenter
(DAL06)
DAL05
Eureka
Local
LBs
Web
App
Auth
Service
Booking
Service
Cluster
Auto
Recovery
and
Scaling
Services
Global
Load
Balancers
Rule
Why?
Always
>
2
of
everything
1
is
SPOF,
2
doesn’t
web
scale
and
slow
DR
recovery
Including
IaaS
and
cloud
services
You’re
only
as
strong
as
your
weakest
dependency
Use
auto
scaler/recovery
monitoring
Clusters
guarantee
availability
and
service
latency
Use
applicaBon
level
health
checks
Instance
on
the
network
!=
healthy
IaaS
High
Availability

7. Our
goals
for
this
talk
¡ Change
your
perspecBve:
as
we
move
into
hosted
offerings,
scalability
and
availability
are
criBcal
to
our
business
¡ We’ll
look
at
NeWlix
RunBme
Libraries
that
you
can
adopt
in
your
products
today
–
regardless
of
whether
you’re
building
a
hosted
service
¡ The
Libraries
promote
good
development
pracBces
and
architectural
paperns

8. Pre-­‐req:
Inversion
of
Control
PaJern
¡ Inversion
of
Control
or
Dependency
Injec4on
is
a
soSware
design
papern
that
defines
a
set
of
callback
interfaces
which
are
provided
with
their
data
or
references
when
needed
¡ The
client
“inverts
control”
back
to
some
container
to
tell
it
what
references
to
use
during
execuBon
¡ Spring,
Java
Persistence
API,
Servlet,
etc
are
all
forms
of
this
papern
¡ NeWlix
leverages
Google
Guice
(hpps://code.google.com/p/google-­‐guice/)
and
makes
heavy
use
of
injecBon
for
their
lifecycle
management
library
(Karyon)

9. 9
IBM Confidential
Lifecycle
Management:
Karyon
¡ In
order
to
support
microservices,
it’s
important
to
componenBze
your
architecture
between
microservices
¡ But
within
microservices,
there
are
similar
logically
independent
segments
–
EmailNoBficaBon,
Database
API,
Storage
API,
etc
¡ If
you
have
a
big
ServletContextListener
today
which
does
a
bunch
of
iniBalizaBon,
pay
apenBon
here
–
this
will
help
you
refactor
that
code
into
a
more
manageable
approach
StorageComponent
DatabaseComponent
EmailNotificationComponent
MyProductApplication
http://…/yourservice
http://…/myservice

10. 10
IBM Confidential
Lifecycle
Management:
Karyon
¡ Karyon
defines
@ApplicaBon
and
@Component
annotaBons
¡ Well
defined,
loosely
coupled
classes
form
the
larger
applicaBon
(e.g.
Components)
¡ Allows
us
to
change
the
monolithic
ConfiguraBonService
class
(1500
lines
originally)
into
about
65
lines
¡ Any
new
parts
of
the
architecture
should
be
defined
as
Components
¡ Also
defines
a
“HealthCheck”
type
to
be
subclassed
for
each
ApplicaBon
package com.urbancode.landscape.web.core;
…
@Application
@Singleton
public class LandscaperApplication {
private static final Logger logger = LoggerFactory
.getLogger(LandscaperApplication.class);
@Inject
private ServletContext context;
@Inject
private LocalStorage storage;
@Inject
private PersistenceConfig persistenceConfig;
@Inject
private ServiceBusClient serviceClient;
@Inject
private NotificationConfig notificationConfig;
@PostConstruct
public void initialize() {
..
}
}

11. 11
IBM Confidential
Karyon:
@Component
¡ @Components
break
up
logical
parts
of
the
larger
applicaBon
(as
expected)
¡ Components
can
declare
other
Components
in
their
InjecBons
(e.g
PersistenceConfig
-­‐>
LocalStorage)
¡ Components
are
iniBalized
**BEFORE**
the
ApplicaBon
(hence
cannot
depend
on
the
ApplicaBon)
¡ Components
may
choose
to
be
@Singleton
¡
Components
may
declare
@PreDestroy
methods
package com.urbancode.landscape;
…
@Component
@Singleton
public class LocalStorage {
@Inject
private ServletContext context;
..
@PostConstruct
public void initialize() {
..
}
…
}

12. 12
IBM Confidential
Karyon
Admin
Console
¡ Available
at
hpp://localhost:8077/
(just
by
launching
Tomcat)
¡ Provides
detailed
informaBon
about
the
process
including
classpath,
environment
variables,
properBes,
and
a
web-­‐based
JMX
console

13. 13
IBM Confidential
Configura7on
Management:
Archaius
¡ Archaius
defines
a
library
for
managing
applicaBon
properBes
¡ Exposed
automaBcally
through
Karyon
admin
console
¡ Allows
you
to
define
“composite”
sources:
¡ pull
from
properBes
file,
¡ then
database,
¡ then
environment-­‐specific
properBes
file
¡ Makes
overriding
properBes
much
easier

14. 14
IBM Confidential
Dynamic
Proper7es
with
Archaius
¡ Archaius
also
defines
dynamic
properBes
¡ Dynamic
property
API
updates
properBes
for
each
request
¡ Allows
you
to
update
configuraBon
without
forcing
a
reboot
¡ ProperBes
can
be
manipulated
by
JMX
at
runBme
// Create the dynamic property
DynamicStringProperty novaEndpoint =
DynamicPropertyFactory.getInstance()
.getStringProperty(“key”, ”default_value");
// each new request will get the latest known value
novaEndpoint.get();

15. 15
IBM Confidential
Latency
&
Fault
Tolerance:
Hystrix
¡ Failure
happens.
Daily.
Hourly.
This
minute
even.
¡ Failure
cannot
be
prevented,
it
can
only
be
protected
against
through
isolaBon
¡ When
one
dependency
fails,
it
can
cause
cascading
failures
or
chain
reacBons
which
have
a
much
broader
impact
¡ Release
It!
describes
many
paperns
and
anBpaperns
around
stability
and
availability
moBvated
by
exactly
this
kind
of
use
case

16. 16
IBM Confidential
Bulkhead
PaJern
¡ When
a
cascading
failure
or
chain
reacBon
occurs,
the
enBre
user
experience
can
be
destroyed
by
one
bad
actor
¡ In
this
case,
or
in
similar
examples
where
one
or
more
dependencies
all
fail
due
to
an
upstream
service,
you
want
to
isolate
that
failure
so
that
it
doesn’t
impact
the
rest
of
the
architecture
¡ We
call
this
protecBon
the
“Bulkhead”
papern

17. 17
IBM Confidential
Circuit
Breaker
PaJern
¡ When
a
failure
occurs
over
and
over
again,
a
backend
system
may
be
down
or
experiencing
too
much
load
¡ In
the
case
of
too
much
load,
you
can
“shed
load”
to
prevent
further
degradaBon
¡ We
can
“trip
the
circuit”,
meaning
that
we
avoid
sending
new
requests
aSer
some
failure
threshold
(default
to
50%)
¡ We
call
this
“Circuit
Breaker”

18. 18
IBM Confidential
Fail
Fast
PaJern
¡ When
a
circuit
is
“open”
(e.g.
the
backend
service
is
failing
consistently),
any
new
requests
are
immediately
rejected
or
the
fallback
mechanism
is
invoked
¡ We
call
this
failing
fast.
¡ Enables
you
to
respond
to
failure
in
a
predictable
way
by
implemenBng
fallbacks
in
each
command

19. 19
IBM Confidential
Encapsula7ng
Service
Dependencies:
HystrixCommand<R>
¡ Each
command
extends
a
common
type
¡ The
constructor
configures
Group
ID,
Command
ID,
and
other
sevngs
¡ A
run
method
implements
the
behavior
public class OSGetAccessTokenCommand extends
HystrixCommand<Access> {
private static final HystrixCommandKey
GET_ACCESS_TOKEN_CMD_KEY =
HystrixCommandKey.Factory.asKey("GetAccessToken”);
public OSGetAccessTokenCommand(…) {
super(Setter.withGroupKey(
HystrixCommandGroupKey.Factory.asKey(KEYSTONE_GROUP_ID))
.andCommandKey(GET_ACCESS_TOKEN_CMD_KEY)
.andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
));
...
}
...
protected Access run() throws Exception {
...
}
}

20. 20
IBM Confidential
More
than
your
money’s
worth
¡ All
commands
with
the
same
Group
ID
have
their
own
resource
pool
(more
on
that
in
a
bit)
¡ Commands
have
configurable
Bmeouts
which
are
automaBcally
enforced
¡ Commands
can
define
fallbacks
in
case
of
Bme
out
or
dependency
failure
¡ Support
opBmizaBons
such
as
Request
Collapsing
and
Request
Caching
Future<Access> request =
new OSGetAccessTokenCommand(
identityEndpoint.get(),
identityUsername.get(),
identityPassword.get())
.queue();
Access credentialToken = request.get();

21. 21
IBM Confidential
In
case
of
emergency
…
¡ Implement
getFallback()
when
failures
occur
¡ OpBons
might
include
returning
mock
data
or
non-­‐personalized
data
¡ OpBonally
check
for
the
failure
reason
with
getFailedExecuBonExcepBon()
(Bmeout,
excepBon,
etc)
¡ When
using
Jersey
for
REST
API,
use
javax.ws.rs.core.Response
as
your
response
type,
and
decide
proper
Response
in
getFallback()
public class UCDGetComponentsCommand extends
UCDAbstractCommand<JSONArray> {
@Override
protected JSONArray getFallback() {
JSONArray componentsArray = new JSONArray();
// Return mock data in case of failure
for (String name : new String[]
{ "JKE Web", "JKE Database",
"Mortgage App", "JPetStore Web" }) {
try {
MockData.getInstance()
.createComponentResource(componentsArray,
name);
} catch (JSONException e) {
…
}
}
return componentsArray;
}

22. 22
IBM Confidential
Methods
of
Isola7on
for
Commands
¡ Hystrix
supports
two
forms
of
IsolaBon:
Thread
Pool
and
Semaphore
¡ Thread
Pool
is
the
easiest
to
understand
–
each
Group
has
its
own
dedicated
Thread
Pool
and
when
requests
come
in
while
the
thread
pool
they
fail
fast
¡ Semaphore
is
useful
though
if
you
want
IsolaBon
but
don’t
want
to
break
your
exisBng
Threading
model
–
example
session
filters
which
configure
ThreadLocals
for
Servlets
or
Jersey
Resources
public abstract class UCLAbstractDBCommand<R> extends
UCLAbstractCommand<R>{
public UCLAbstractDBCommand(
HystrixCommand.Setter setter) {
super(setter.andCommandPropertiesDefaults(
HystrixCommandProperties.Setter()
.withExecutionIsolationStrategy(
ExecutionIsolationStrategy.SEMAPHORE)));
}
@Override
protected final R run() throws Exception {
return doRun();
}
# Runs in the Jersey Resource’s thread
# Leverages ThreadLocals from Hibernate Session Filter
protected abstract R doRun() throws Exception;
}

23. 23
IBM Confidential
When
to
throw
in
the
towel
…
(configuring
7meouts,
etc)
¡ Virtually
all
of
the
HystrixCommand’s
opBons
can
be
configured
at
runBme
through
Archaius
¡ Notable
examples
include
the
Thread
Pool
limits
and
Bmeouts
¡ See
the
Hystrix
wiki
for
a
full
accounBng
of
what
you
can
do
¡ Set
these
properBes
dynamically
through
Karyon’s
JMX
console
and
watch
command
behavior
change
on
the
fly
# landscaper.properties or landscaper-[env-name].properties
# configure default timeout milliseconds
hystrix.command.GetAccessToken.execution.isolation.thread.ti
meoutInMilliseconds=20000
hystrix.command.OSGetImages.execution.isolation.thread.tim
eoutInMilliseconds=8000
…
# let no more than 20 GetAccessToken commands run
# together
hystrix.threadpool.KeystoneGroup.maxQueueSize=20
# wait 15 seconds if we “trip” the circuit for a command
hystrix.command.GetAccessToken.circuitBreaker.sleepWindo
wInMilliseconds=15000

24. 24
IBM Confidential
Embedding
Unit
Tests
¡ Promoted
approach
by
NeWlix
to
reduce
fricBon
and
introduces
limited
addiBonal
bytes
relaBve
to
third
party
libraries
¡ Makes
it
easy
to
write
commands,
the
UnitTest
becomes
the
test
harness
and
verificaBon
¡ Always
testSuccess()
and
testFailure()
use
cases
to
ensure
expected
behavior
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
public class OSDeployEnvironmentCommand extends
OSAbstractOrchestrationCommand<Response> {
public static class UnitTest {
@Test
public void testSuccess() throws InterruptedException,
ExecutionException, JSONException {
…
Future<Response> request =
new OSDeployEnvironmentCommand(…)
.queue();
Response response = request.get();
assertNotNull(response);
assertEquals(HttpStatus.SC_OK,
response.getStatus());
}
}

25. 25
IBM Confidential
Bootstrapping
Karyon
–
Advanced
Topic
¡ If
your
commands
leverage
Karyon,
it’s
possible
to
bootstrap
Karyon
as
part
of
a
UnitTest
¡ Requires
a
liple
more
setup,
but
enables
you
to
have
control
over
the
dependency
injecBon
from
the
Google
Guice
container
¡ Allows
you
to
create
Mock
classes
for
things
like
ServletContext
or
other
classes,
if
your
Jersey
Resource
or
Hystrix
Command
needs
them
protected static KaryonServer karyonServer;
protected static Injector injector;
protected static void configureKaryon(List<String>
packages) throws Exception {
if (karyonServer == null) {
ConfigurationManager
.loadPropertiesFromResources(
"landscaper-heat-unittests.properties");
…
karyonServer = new KaryonServer();
injector = karyonServer.initialize();
karyonServer.start();
…

26. 26
IBM Confidential
REST
API
with
Ribbon
¡ A
microservice
architecture
is
generally
built
around
REST-­‐
based
interfaces
¡ Ribbon
provides
a
HTTP
client
for
calling
service
dependencies
¡ Provides
more
visibility
into
behaviors
like
connecBon
Bmeouts,
auto-­‐retry
and
number
of
retries
through
Archaius
properBes
¡ Also
supports
client-­‐side
load
balancing
through
Eureka
ConfigurationManager
.loadPropertiesFromResources(
"heat-api-client.properties");
ClientFactory.getNamedClient("heat-api-client");
URI uri = new URI("/v1/" +
token.getToken().getTenant().getId()
+ "/stacks");
HttpRequest request = HttpRequest.newBuilder().uri(uri)
.verb(Verb.POST)
.entity(getEntity().toString().getBytes())
.header("Content-Type", "application/json")
.header("Accept", "application/json")
.header("User-Agent", "python-heatclient")
.build();
HttpResponse response =
heatAPIClient.executeWithLoadBalancer(request);

27. 27
IBM Confidential
Tying
it
all
into
a
bow
…
¡ Ribbon
is
generally
used
with
HystrixCommands
to
execute
requests
¡ Ribbon
will
automaBcally
discover
available
servers
and
load
balance
between
them
¡ Here
the
example
has
an
explicit
list
of
servers,
but
we
can
change
the
sevngs
to
use
Eureka
for
service
discovery
# configuration settings heat-api-client.properties
# Interval to refresh the server list from the source
heat-api-client.ribbon.ServerListRefreshInterval=2000
# Connect timeout used by Apache HttpClient
heat-api-client.ribbon.ConnectTimeout=3000
heat-api-client.ribbon.listOfServers=localhost:
8004,10.0.2.15:8004
// No need to reference an explicit server, Ribbon finds one
URI uri = new URI("/v1/" +
token.getToken().getTenant().getId()
+ "/stacks");
HttpRequest request = HttpRequest.newBuilder().uri(uri)
.verb(Verb.POST)
...
HttpResponse response =
heatAPIClient.executeWithLoadBalancer(request);

28. 28
IBM Confidential
Leveraging
Hystrix’s
Built
In
Event
Stream

29. 29
IBM Confidential
Netflix OSS IBM port/enablement
Netflix “Zen” of Cloud • Worked with initial services to enable cloud native arch
• Worked with initial services to enable NetflixOSS usages
• Created scorecard and tests for “cloud native” readiness
Highly Available IaaS and Cloud
Services
• Deployment across multiple IBM SoftLayer IaaS datacenters and global and local
load balancers
• Complete automation via IBM SoftLayer IaaS API’s
• Ensured facilities for automatic failure recovery
Micro-service Runtimes (Karyon,
Eureka Client, Ribbon, Hystrix,
Archaius)
• Ported to work with IBM SoftLayer IaaS and on the WebSphere Liberty Profile
application server
• Created “eureka-sidecar” for non-Java runtimes and ElasticSearch discovery
Netflix OSS Servers
(Asgard, Eureka Server, Turbine)
• Ported to work with IBM SoftLayer IaaS + RightScale
• Operationalized HA and secure deployments for multiple service tenants
Adopted Chaos Testing • Ported Chaos Monkey to IBM SoftLayer IaaS
• Performed manual Chaos Gorilla validation on services
Worked through devops tool
chain
• Worked with initial services to enable continuous delivery with devops (and imagine
baking via Animator like tool)
• Working through integration with Urban Code Deploy and other IBM continuous
delivery tools
AIM
Scalable
Services
Fabric
Ne0lixOSS
Port
to
So_Layer

30. 30
IBM Confidential
Demo
of
Ne0lixOSS
and
Ne0lix
services
on
So_Layer
Region (Dallas)
DAL01
Datacenter (DAL06)
DAL05
Eureka
Local LBs
Web App Auth Service Booking Service
Cluster Auto Recovery and Scaling Services
Global Load
Balancers
Asgard
SoftLayer GLB/LLB and Datacenters

31. 31
IBM Confidential
Demo
of
Ne0lixOSS
and
Ne0lix
services
on
So_Layer
Turbine
Eureka

32. 32
IBM Confidential
About
Your
Architecture
• Architecture should support DevOps principles such as staged roll out, operational insights, and scriptability
• Each resource provides some very practical advice for building systems which are focused on reliability and
feedback loops
Release It!: Design and
Deploy Production-Ready
Software
http://netflix.github.io/#repo

33. 33
IBM Confidential