Applications are moving to cloud and containers to boost reliability and speed delivery to production. However, if we use the same old approaches to testing, we'll fail to achieve the benefits of cloud. But what do we really need to change? We know we need to automate tests, but how do we keep our automation assets from becoming obsolete? Automatically provisioning test environments seems close, but some parts of our applications are hard to move to cloud.

1. Modernizing Testing
as Apps Re-Architect
Al Wagner
Technical Evangelist
IBM Cloud DevOps
awagner@ca.ibm.com
@Al_Wagner
Chris Haggan
Product Manager
HCL Technologies
christopher.haggan@hcl.com
@mrchris2000

2. Flexibility Agility
2
Adaptive
Automation
Digital
Transformation
Embracing cloud is key to
a modernization journey

3. The three ways…
Lift and Shift
Rebuild
Refactor

4. Lift & Shift RebuildRefactor
De-mystifying the three ways
Other dependent systems,
applications, services
Other dependent systems,
applications, services
Other dependent systems,
applications, services
Other dependent systems,
applications, services
API / Rest API / Rest
API / Rest API / Rest

5. App
M/W
Data
OS
Physical Hypervisor
“Uber” distributed applications…
… on the journey to Cloud native!
Increasing Flexibility; Efficiency; Strategic Value
Containers Container Orchestration
Master
etcd Controller
API Server Manager
Scheduler
Node 1
Node 2
Node 3
kubelet
kubelet
kubelet
Pod Pod
Pod Pod
Pod Pod
Cloud
(public/private)
(Java, ERP, zOS)

6. Industrialized Core (Systems of Record)
Traditional IT
On premises systems
z, i, Java based
deliver at regular cadence • waterfall/agile • stability •
predictability • lean delivery pipeline • core & legacy
Dev SCM Build Repo Test Stage Production
Dev SCM Build Repo Test Stage Production
Dev SCM Build Repo Test Stage Production
DevOps in a multi-speed world
Innovation Edge (Systems of Engagement)
Cloud Native
Born on the Cloud Systems
rapid delivery for innovation • Agile • anti-fragile •
experimentation • new and innovative
Mobile Device
Mainframe
Traditional
Public
Private
Production
API
-----
Rest

7. What does an end to end test environment require?
If you had what you needed:
4 How much faster would your
development processes be?
4 How much more effective would your
testing become?
4 How much more value could you get
from staff who were no longer waiting
for environments?
4 How much risk could you remove from
each deployment into production?
Third Party
APIs
Unpredictable
dependencies
Available
dependencies
Application
Under Test
Physical
devices
Predictable
Functionality
Production
Systems
Applications
from other
teams

8. Ideally, continuous testing means continuous
feedback, so that problems can be addressed
quickly.
However, many testers need to wait for
environments to become available…
So if your team is waiting for environments
instead of testing:
• How fast is your feedback to development?
• How valid is that feedback?
• And have you built new code on top of
defective code?
Waiting for environments
Design & Build
Deployment to test
environments
API Testing
Performance Testing
UI Automation

9. Test infrastructure on demand
Dev SCM Build Repo Test Stage Production
Dev SCM Build Repo Test Stage Production
Dev SCM Build Repo Test Stage Production
Product A
Team
Product B
Team
Product C
Team
Infrastructure
Product Team
(DevOps Engineers / SR Engineers)
Provision, Deploy, Test, and Manage
ü Virtual services
ü Test data
Infrastructure as code:
ü Multi cloud management
ü Terraform configurations
ü Environment blueprints
ü Middleware configurations
ü Helm charts
ü API gateway
ü Monitoring
ü Security policies
Repo

10. How are you
testing
software
today?
Application
complexity and
areas of elevated
risk are below the
surface!
<10%
90%

11. A smarter
approach -
ContinuousT
esting
Automate integration testing
to increase coverage and
confidence
Earlier automated
performance testing allows
faster resolution of
architectural issues
Use of Service Virtualization
to eliminates delays due to
unavailable software across
any layer
`
Exploratory
Testing
Component
Layer
Service / API
Layer
UI
Layer
Push testing
down to focus on
the highest areas
of risk
V
i
r
t
u
a
l
S
e
r
v
i
c
e
s
Isolate defects for
easier & faster
remediation

12. “Let’s write a mock!”
!

13. Programmers should be
writing revenue generating
code instead of creating and
maintaining mocks, stubs, or
fakes!
DevOps is about delivering
value to the business.
The essence of lean is to
eliminate waste.
“Let’s write a mock!”
!

14. Reuse APM data to provide
more realistic response times
No need to reconfigure
environments when adding or
removing virtual services
Handle specified scenarios
with virtual services, while
directing others to the live
system
A wide variety of protocol and
message support, which can
be mixed and matched as
needed
Virtualize dependencies to
accelerate development and
testing
Rational Test Workbench / Rational Test Virtualization Server
Tools to aid with the creation
of both simple and complex
virtual services

15. Avoiding the Big Bang with Incremental Integration Testing
Incremental Integration Testing
üAs services become available, they can be introduced to the system.
üServices, applications, systems are introduced into the continuous
integration cycle in a prioritized, controlled fashion.
Actual service
Virtual service

16. Analyse the system
to find endpoints and
understand the
message traffic
between services
Virtual Service Creation
Build a model of the
system under test,
enabling reuse of
test assets across
environments
Use the model to
create powerful
codeless virtual
services for a wide
variety of
technologies
Publish virtual
services to be reused
by other teams
within the enterprise
Discover Model Virtualize Share

17. 4 Reduce testing time by running
test scripts in parallel, across
different browsers
4 Go further by running test
scripts in parallel using Docker
containers, and take advantage
of cloud capabilities and run
your test scripts in ICP
Deploy to IBM Cloud Private
Gain scale and speed advantages
Accelerate Functional Testing

18. Use multiple capabilities to
discover how the system
under test fits together:
• Recording (Web and
Thick client)
• Page Object Models
• Object Repositories
UI Tests - Creation and Deployment
Apply rules and requirements
to the application test
definition:
• Live page
representations
• Custom validations
Apply test data and execute:
• Advanced datapool
technology enables data
driven solutions
• Dynamic data provided
on-demand
• Local execution to
validate test flow
Use UI Test execution in
containers to run UI tests
against on-premise, hybrid or
cloud native applications
• Elastic capacity through
ICP
• Fast execution time in
containers
• Results and reports
centralised
Define Elaborate Run Scale

19. Performance Verification
4 Use a visual workload model to define
performance stages and durations
4 Observe the effect of load on
individual components with
integrated reporting
4 Use bare metal, virtual machines,
Docker containers, IBM Cloud Private
resources or a combination to rapidly
scale load generation
Client-Side Performance
Controller
Performance Load Generators
System Under Test

20. Performance Verification
4 Gold Master technology allows
performance tests to be updated
automatically using UI templates
4 Low Impact performance reporting
enables test engineers to gain
insight into potential problem areas
sooner
4 Advanced resource monitoring
capabilities identify problems in
your applications not just the
hardware and Operating System

21. Use multiple capabilities to
discover how the system
under test fits together:
• Recording (Web and
Thick client)
• Page Object Models
• Object Repositories
Performance Tests - Creation and Deployment
Apply rules and requirements
to the application test
definition:
• Live page
representations
• Custom validations
Advanced resource
monitoring of the system
under test:
• Agentless resource data
capture
• Define and apply SLAs for
performance
requirements
• Set resource usage goals
and validate
Use Performance Test
execution in containers to
run performance tests
against on-premise, hybrid or
cloud native applications
• Elastic capacity through
ICP
• Fast execution time in
containers
• Results and reports
centralised
Define Elaborate Monitor Scale

22. 22 |
Testing of applications through
terminals, browsers and mobile
interfaces.
Our Continuous Testing Solution
Functional Testing
Create realistic workloads on your
systems, track SLAs, and analyse
the root causes of any issues.
Trigger and validate a system’s
APIs across a range of network
protocols and message formats.
Virtualize missing components of
the system to enable faster
development and testing.
Performance Testing API Testing Service Virtualization
{
“message”:{
“key1”:”value1”,
“key2”:”value2”
}
}
Rational Functional Tester
Rational Performance
Tester
Rational Integration Tester
Rational Test
Virtualization Server
Rational Test Workbench

23. Develop
Check in
Build
code
Deploy
new
software
Deploy
virtual
services
Test &
Provide
Feedback
{dev}
Test Automation and DevOps
Continuous Integration
Continuous Delivery
Continuous Testing

24. Bringing it all together
IBM Rational Test
Virtualization Server
IBM UrbanCode
Deploy
IBM Rational
Test Workbench
integrated with
Deployment Automation
Data Internal
Messages
3rd-party
Services
virtual components
Dev QA
Test Environments
Dynamic Infrastructure
§Deploy what is ready,
virtualize the rest
§Continuously test in
production-like env
§Right test data at
the right time
IBM Optim Test Data
Orchestrator
UC Deploy Plugins
IBM Rational
Quality Manager §Manage quality across the
delivery lifecycle
IBM UrbanCode
Build
Continuous Integration Continuous Delivery
jUnit
nUnit
zUnit
MSTest
CppUnit
IBM UrbanCode Release or Velocity
Continuous Tracking and Planning
Continuous Testing
IBM Rational Team Concert
Impact Analysis
Release Orchestration
Release Orchestration
Work Item Mgmt
Version Control
Build
Mgmt
Production
Mainframe
Traditional
Cloud
Mobile Device
• Rational
Application
Developer
• IBM Developer
for Z Systems
• Eclipse
• Orion
• Etc…

25. Achieving DevOps agility with the mainframe
Aiming to bring faster development and testing to its
mainframe environment without sacrificing stability and
scalability, Rabobank uses IBM DevOps solutions to
automate end-to-end tests.
“For our business, this means that we have much more agility. We could bring changes to the
business in a much higher speed, while at the same time safeguarding that they still perform.
That they don’t mess up the stability of our systems.”
— Ralph Van Beek, DevOps Architect, Rabobank
ü 10 to 15x faster testing than possible when relying on manual configuration
ü 60 end-to-end test cycles per month
ü Greater business agility through much faster application changes

26. Come together and think
Think is the business and technology event of
the year.
Join us to create in the cloud, innovate with IoT, break down barriers
with blockchain, and solve the unsolvable with quantum. Learn how to
get more value from AI and your data—train it, trust it, and make your business
more efficient and secure.
See how the newest technology paired with our services and industry expertise
can help reimagine your business and industry.
February 12 – 15
Moscone Center
San Francisco, CA
Register today at ibm.com/think

27. Thank you