Most major enterprises have invested millions of dollars on soIware performance lab infrastructure, that develop recurring maintenance effort and costs, unstable environments, and conflicts over constrained resources. To combat this drain and deliver value, companies need to find ways to optimize. Virtualizing the behavior and performance characteristics of test lab dependencies, CA LISA Service Virtualization has helped many enterprises achieving highly available labs that allow earlier performance testing with greater flexibility, at a much lower cost. Techniques discussed in this session include: Removing capacity constraints from performance testing lab Optimizing performance management by decomposing SLAs Shift-Left: conducting incremental, iterative performance testing

1. Service
Virtualiza.on:
Reality
is
Overrated.
a
CA
LISA
Service
Virtualiza.on
Best
Prac.ce:
Transform
Performance
and
Scale
Min
Fang
min.fang@ca.com
+65
9636
8356
NGTC
Conference
(c)

2. Agenda
•
•
•
•
Constraints:
The
Enemy
of
Agility
What
is
Service
Virtualiza.on
Capabili.es
of
Service
Virtualiza.on
Best
Prac.ce:
Transform
Performance
&
Scale
Next
Genera.on
Tes.ng
Conference
(c)

3. Accelera.ng
Applica.on
Time-­‐to-­‐Market
HOW
DO
WE
INCREASE
“TOUCH
TIME”
AND
REDUCE
“IDLE
TIME”?
DEVELOPMENT
REGRESSION
QA
INTEGRATION
CHALLENGES
§ Constraints
of
dependent
systems
needed
to
move
the
SDLC
forward
§ Lack
of
CollaboraFon
creates
scheduling
conflicts
and
buffers
from
development
through
opera.ons
§ Increasing
Complexity
of
today’s
heterogeneous
enterprise
IT
architectures
§ Need
Complete
visibility
into
what
happens
in
produc.on
PRODUCTION
UAT
PERFORMANCE

4. Bridging
the
Gaps
in
DevOps
COLLABORATIVE
PORTFOLIO
PLANNING
OPERATIONS
MONITORING
RELEASE
AUTOMATION
DATA
MINING
SERVICE
VIRTUALIZATION
AGILE
DEVELOP
&
TEST
ENVIRONMENT
MANAGEMENT
CONTINUOUS
FEEDBACK
PLAN
BUILD
RUN
DevOps
brings
the
applica.on
porbolio
forward
to
meet
market
challenges
with
collabora.on
and
efficiency
from
design
to
deployment

5. Constraints:
The
Enemy
of
Agility
NGTC
Conference
(c)

6. 1.
Insufficient
environment/capacity
for
tes.ng
and
development
UAT
Tester
SOAP
Performance
Tester
AppServer
MainFrame
100%
u.liza.on
Developer
Copyright
©
2010
CA.
All
rights
reserved.

7. 2.
Uncontrollable
environment
growth
SOAP
UAT
Tester
AppServer1
SOAP
Dev
Team1
AppServer2
Keep
adding
MIPS
to
keep
pace
with
new
requirements
SOAP
Training
MainFrame
AppServer3
Copyright
©
2010
CA.
All
rights
reserved.

8. 3.
System
Dependencies
=
Development
&
tes.ng
constraint
Tester/
Developer
SYSTEM
UNDER
TEST
Frontend
MainFrame
1.
Upgrading
MQ
version
ESB
3.
ESB
Team
will
only
complete
development
in
2
weeks
.me
2.
Timesharing
Mainframe

9. What
is
Service
Virtualiza.on
NGTC
Conference
(c)

10. Service
Virtualiza.on:
How
does
it
work?
CAPTURE
PROCESS
MODEL
R Structured Conversations
R Heuristics
R Sophisticated Behavior
R Observe
R Analytics
R Understand
R Algorithms
R Dynamic Properties (Dates,
values, etc.)
R Protocol-Level
R State
R Scenario Support
§ Recorded traffic
R Test Data Sanitation
§ Design specs
R Compiled Model vs. Stubs
§ Sample RR pairs
R Automatic Healing
§ Transaction logs
§ Manual Creation
§ Byte-Code

11. Captures
and
Models
Realis.c
Virtual
Services
Legacy
SOAP
ERP
Source
ApplicaFon
Capture
Database
Create
Virtual
Services
JDBC
MQ
§ Automa.cally
capture
realis.c
behavior
§ Op.mize
models
as
desired
Mainframe

12. LISA
Virtualizes
en.re
test
environments
to
eliminate
costly
constraints
Legacy
SOAP/HTTP
Source
ApplicaFon
IT
resources
now
available
on
your
terms,
accelera.ng
.me-­‐to-­‐market
and
team
produc.vity,
allowing
24/7
tes.ng
MQ
JDBC
ERP
VS1
DB1
VS2’
Virtual
Test
Environments
Database
Mainframe
W
W
W
.
I
T
K
O
.
C
O
M

13. CA
LISA
Virtual
Service
Environments
Customers
can
virtualize
almost
any
IT
asset
§ Mainframes
&
Mainframe
Components
CICS-­‐CICS
DB2
MQ
§ Plaborms
§ SOA
Protocols
XML
SOAP
REST
HTTP
§ Risk
Solu.on
Providers
§ Credit
Bureau
§ SMS
Gateway
Providers
§ Bill
Payment
Services
§ Card
Processing
Services
§ Teleconference
Service
Provider
§ Travel
System
(GDS)
JMS
MQ
UDDI
.NET
CORBA
JDBC
JAVA

14. How
Can
Service
Virtualiza.on
Help?
NGTC
Conference
(c)

15. 1.
Model
Applica.on
Under
Development
Source
App
Virtual
service
can
be
modeled
from
design
ar.facts.
Allowing
development
and
integra.on
test
to
start
earlier
Virtual
Service
Model
Design
arFfacts,
transacFon
logs,
WSDL
files,
flat
files,
and
others

16. 2.
Using
SV
for
training
environment
(Use
Client
without
Backend)
MainFrame
AppServer
Virtualize
AppServer

17. 3.
Virtual
Data
Management
-­‐
For
Development
&
Tes.ng
Applica.on
Server
Virtual
Service
Environment
Abu
Bakar
IC:
1234
Acc:
M123
Alvin
Koh
IC:
4567
Acc:
456
Arun
Kailasam
IC:
54354
Acc:D4345
You
have
full
control
over
data
in
virtual
service,
you
can
add
your
data
directly
into
the
virtual
service
for
the
purpose
of:
a) Scenario-­‐based
tes.ng
b) Nega.ve
tes.ng
Copyright
©
2010
CA.
All
rights
reserved.

18. 3.
Virtual
Data
Management
-­‐
Create
or
Modify
virtual
data
directly
3.
Nega.ve
Tes.ng:
How
do
my
applica.on
handle
-­‐10k
as
account
balance
1.
If
AccountID=
=1313094687566198
2.
Return
user
details
of
JACKY
LEE
(exis.ng
customer)

19. 4.
Using
SV
for
Performance
Engineering
1.
Interface
load
tes.ng
2.
Load
tes.ng
without
u.lizing
mainframe
MIPS
Test
Test
Test
1000
VU
(Interface)
SOAP
Virtualize
AppServer
3.
At
the
same
.me,
understand
the
txn/sec
requirement
and
to
size
your
mainframe
more
accurately
100
tps
(Virtual
Mainframe)

20. Aligning
to
CIO
Goals
Do
More
with
Less
Number
of
Test
Environments
Number
of
Training
Environments
Complexity
in
the
environments
4
#1
consumer
retail
bank
of
America
eliminated
$100M
in
hardware
requirements
when
using
LISA
over
5
years.
Over
10x
ROI,
6x
in
year
1
3.5
3
2.5
2
Normal
1.5
1
0.5
CIO
Goal
accomplished!
CONTROL
costs
0
Year
0
Year
1
Year
2
Year
3
Year
4
Year
5
Year
6
Year
7
Halt
growth
by
using
LISA
3Y
TCO:
Reduce
overall
future
footprint
by
50%
or
more
5Y
TCO:
Reduce
overall
future
footprint
by
70%
or
more

21. Best
Prac.ce:
Transform
Performance
&
Scale
NGTC
Conference
(c)

22. When
customers
“struggle”
with
an
app…
In
the
.me
it
takes
to
blink
an
eye—400
milliseconds
to
be
exact,
online
AND
mobile
users
will
have
begun
to
abandon
your
applica.on
Your
compeFtor
You
43%
Abandon
the
transac.on,
try
later
on
computer
(Harris
Interac.ve
Mobile
Transac.ons
Survey
for
TeaLeaf,
March
2011)
16%
Become
more
likely
to
prefer
a
compe.tor
14%
Log
a
complaint
with
customer
service
12%
Go
directly
to
a
compe.tor

23. Performance
will
make
or
break
an
applica.on!.
When
problems
occur:
Development
Phase
• Revenue
is
lost.
Requirements
• Produc.vity
decreases.
• Customers
leave.
• The
longer
it
takes
to
find
the
defect,
the
more
it
Costs
to
fix.
Cost
of
Fixing
RaFo
1
Design
3
-­‐
6x
Coding
10x
Development
Tes.ng
15
-­‐
40x
Acceptance
Tes.ng
30
-­‐
70x
Opera.on/Produc.on
40
-­‐
1000
80%
of
the
TCO
of
an
applica.on
is
directly
related
to
finding
and
fixing
problems.
(Source:
“The
Economic
Impacts
of
Inadequate
Infrastructure
for
So<ware
Tes>ng,
NIST)
The
result:
on
average,
a
performance
issue
costs
an
organiza.on
at
least
$45,000/hour.
(Source:
Mul>ple
industry
analysts
have
published
es>mates
ranging
from
EMA’s
$45,000/hr
cost
to
Yankee
Group’s
and
Meta’s
es>mate
of
over
$90,000/hr
or
more
depending
on
the
industry
and
nature
of
the
failing
applica>on.)

24. Service
Virtualiza.on:
Where
would
I
use
this?
Service
Virtualiza.on

25. Eliminate
Capacity
Constraints
Major
North
American
Bank
Performance
&
Test
CoE
VSE
VSE
Shared
Mainframe
Service
VSE
Or
similar
heavy-­‐weight
implementa.on
environment
Before
A{er
Business
Problem
•
•
•
•
•
13
performance,
dev,
test
teams
forced
to
.meshare
on
mainframe-­‐based
services
Cost
of
addi.onal
environments
for
L&P
lab
and
the
movement
to
Agile
development/tes.ng
Difficult
to
perform
load
tes.ng
due
to
system
dependencies
Only
2
of
5
major
releases
were
performance
tested
Offshore
teams
unable
to
complete
off
hours
performance
tes.ng
due
to
availability
issues
Business
Impact
• Virtualize
in
8
days
what
took
2
years
of
custom-­‐coded
stub
development
to
create.
• Virtualize
new
systems
in
1
hour
with
business
analysts
versus
weeks
of
custom
coding
•
•
•
Virtualized
environments
enabled
all
5
major
releases
plus
5
minor
releases
to
include
L&P
tes.ng
Over
$30
million
in
hardware
cost
avoidance
Offshore
teams
performing
300%
more
L&P
tes.ng
due
to
system
availability

26. Customer
Success
Story
Avoid
costly
transacFon
fees
when
tesFng
third-­‐party
services
Leading
Global
Airline
Challenge
§ Test
and
valida.on
of
reserva.on
booking
system
was
incurring
$1
to
2
million
in
monthly
fees
to
access
live
third-­‐party
services
SoluFon
using
LISA
Virtualize
“We
rely
heavily
on
LISA
to
automate
the
tes?ng,
configura?on
and
virtualiza?on
aspects
of
our
test
and
development
environment.”
-­‐
Lead
Architect
§ LISA
simulated
the
behavior
of
third-­‐party
reserva.on
systems
as
virtual
services
that
were
available
24/7
without
usage
fees
§ No
longer
needed
to
coordinate
test
schedules
or
mock
transac.on
data
sets
with
third-­‐party
service
providers
§ Provided
freedom
for
development
and
QA
teams
to
test
earlier
in
the
so{ware
lifecycle
Client
Results
§ Reduced
service
usage
fees
to
save
over
$12M
annually
§ Lower
tes?ng
costs
by
using
stable,
virtualized
data
sets
§ Higher
quality
by
encouraging
earlier
and
more
frequent
tes.ng

27. Example:
Performance
Tes.ng
Middle-­‐Tiers
Using
LISA’s
Virtual
Service
Environments
(VSEs)
USERS:
Agents
/
Adjusters
19.0s
Booking
business
service
LISA
Test
XML/MQ
Conven.onal
load
tes.ng
can
only
expose
overall
performance
.me
UI
Load
Test
Tools
2.5s
3.3s
LISA
Test
Integra.on
Bus
(ESB)
SOAP/HTTP(S
LISA
VSE
PNR
Service
Frequent
Flyer
System
XML/MQ
ROI
Reserva.on
Service
GDS
LISA
VSE
SOAP/JMS
2.5s
LISA
manages
Test
data
LISA
Test
Ticke.ng
Applica.on
LISA
VSE
10.7s
Isolate,
test
and
tune
components
by
virtualizing
dependent
services
Slow
responding
transac.on
or
component
pinpointed
with
LISA

28. Customer
Success
Story
Enabling
High
Performance
with
Constrained
Resources
&
Time
Leading
P&C
Insurance
Fund
Challenge
§ Enterprise
wide
“go-­‐live”
of
cri.cal
quo.ng
system
across
50
states
§ Performance
Tes.ng
lags
could
not
be
pinpointed
with
UI
tes.ng
§ Middle-­‐.er
and
3rd
party
credit
systems
too
cri.cal/costly
to
test.
“When
you
can
do
isola?on
at
a
component
level
against
the
performance
budget,
you
can
iden?fy
the
weak
link
in
the
chain
and
address
it..”
-­‐
Sr.
Director,
IT
Architecture
&
Performance
SoluFon
using
LISA
Virtualize
&
Test
§ Simulated
Web,
IBM
MQ/ESB,
Cloud/3rd
party
Credit
service,
and
Oracle
apps
with
Virtual
Services
for
24/7
inexpensive
tes.ng
§ Drive
50K
user+
load
tests
directly
against
isolated
components
§ “Prove
It”
quick
ROI
project
with
iTKO
engineers
and
SI
partner
Client
Results
§ Went
from
test
overruns
to
unlimited
Virtualized
performance
test
access,
and
star.ng
next
phase
4
months
early
§ Delivered
ROI
within
one
month
with
skilled
resources
§ Improved
test
bed
availability
by
64%
and
increased
number
of
tests
run
by
10x

29. Best
Prac.ce:
Decompose
Service
Levels
Increasing
efficiency
by
building
up
to
meet
a
“Performance
Budget”
Without
component-­‐level
performance
tesFng:
Try adding more test Hardware!!
"
SLA
2.1 seconds"
Order"
Lookup"
Test: "
4.0"
Seconds"
Solution"
Price"
Timeline
$!
8 mo.
"
Test"
3.2
Seconds"
$!
Still not within SLA!!
9 mo.
PROJECT DUE!
Isolated
component-­‐level
performance
tesFng
miFgates
risk:
Order"
0.5s"
0.5s"
Lookup"
Price"
0.4s"
1.9s"
0.4s"
0.5s"
1.2s"
Use VS to isolate & tune components
separately from dependencies!
1.2s"
Solution"
"
Test:"
2.1
Seconds"
Aggregate solution tuned
for performance budgets!

30. Produc.on
vs.
Test:
in
produc.on
4G
SingTel
Singapore
online
banking
shared
infrastructure
datacenter
–
Kuala
Lumpur
web
service
transac.ons
3G
Celcom
Kaula
Lumpur
2.5G
Telkomsel
Jakarta
Mobile
TransacFons
mobile
banking
shared
infrastructure
datacenter
–
Singapore
web
service
Singapore
Kaula
Lumpur
Jakarta
online
banking
shared
infrastructure
datacenter
–
Singapore
Internet
TransacFons

31. ProducFon
vs.
Test:
in
test
without
network
virtualizaFon
Simulated
web
services
Load
Generator
with
network
virtualizaFon
Simulated
TransacFons
online
banking
shared
infrastructure
datacenter
–
Singapore
RealisFc
Mobile/Web/
Internet
Network
CondiFons
4G
SingTel
Singapore
Singapore
3G
Celcom
Kaula
Lumpur
Kaula
Lumpur
Load
Generator
2.5G
Telkomsel
Jakarta
mobile
banking
shared
infrastructure
datacenter
–
Singapore
TransacFons
with
real
world
network
condiFons
Load
Generator
Jakarta
virtual
locaFons
online
banking
shared
infrastructure
datacenter
–
Singapore

32. Xyz
Bank
iPad
Edi.on
Mobile
App
Summary
Analysis
Time
KBs
Data
80
72.2
70
60
50
46
32.8
40
30
20
10
0
44.6
21.3
20.7
14.6
7.1
17.2
33.2
27
22.4
20.5
12.3
11.7
31.9
19.1
8.2
WiFi
3G
3G
Busy

33. Performance
Op.miza.on
-­‐
once
bo}lenecks
iden.fied
than
solu.on
is
to
op.mize
the
applica.on.

34. References
§ Service
Virtualiza.on
Site
Home
Page
:
h}p://servicevirtualiza.on.com/profiles/blog/list?tag=best+prac.ces
§ The
Book
-­‐
Service
Virtualiza.on:
Reality
is
Overrated.
h}p://www.amazon.com/Service-­‐Virtualiza.on-­‐Reality-­‐Is-­‐Overrated-­‐ebook/dp/
B009HAKER8/ref=dp_kinw_strp_1
§ Ge€ng
Started
with
Service
Virtualiza.on
h}p://servicevirtualiza.on.com/video/ge€ng-­‐started-­‐with-­‐service-­‐virtualiza.on-­‐sv-­‐
minute-­‐2012
§ LISA
Introduc.on
–
What
Does
LISA
Do
?
h}p://www.youtube.com/watch?v=2Nl69c3MLkQ
Source
:
CA
Press
Release
|
ServiceVirtualiza.on.com

35. Thank You!
MIN
FANG
MIN.FANG@CA.COM
+65
9636
8356
NGTC
Conference
(c)