Soasta | CloudBees webinar 11/12/2015

Continuous Performance Testing and
Continuous Delivery with Jenkins
November 12, 2015
Tom Chavez, SOASTA @TomChavez
Brian Dawson, CloudBees @brianvdawson

1© 2015 SOASTA Inc.- All rights reserved.
In This Webinar
TODAY’S PRESENTERS
Tom Chavez: SOASTA, Sr. Product Marketing Manager
Brian Dawson: CloudBees, DevOps Practitioner
TWO COMPANIES, A SINGLE SOLUTION
SOASTA: The leader in Mobile and Cloud Testing
CloudBees: The Enterprise Jenkins Company
Together, we offer a complete cloud-based service:
Continuous Performance Testing & Continuous Delivery with Jenkins

Poll Question
What best describes your status with Continuous Integration / Continuous
Delivery (CI/CD)? (single answer)
a) Practicing CI
b) Practicing CD
c) Implementing CI/CD
d) Determining if CI/CD is a good option
e) What is CI/CD?

Agenda
• Why should we performance test earlier?
• Building tests and preparing the test environment
• Connecting your performance tests to your CI (Jenkins)
• Establishing and managing to a performance baseline
• Execute tests in parallel with CD pipelines built and executed
with Jenkins Workflow
• Q&A
Questions:
Submit in the question box during event

Why Should We Load Test Earlier?

Why DON’T We Load Test Earlier?
“Because it’s hard.”
o Tools require coding skills we don’t have (C, VB?)
o We don’t have a scalable test environment
o We don’t have hardware for load generation
o It’s a different tool
o There are too many other tests to do

“Because it takes too long.”
o Scripting takes forever
o Results are slow
o Fast setup of environments is impossible
o We’re dependent on IT or other teams
o We’re too agile

o Results are slow
o We’re too agile
“Because no one ever told us to.”
o There are no “performance coverage” requirements
o There isn’t a performance baseline for reference
o There isn’t a defined performance target
o No one is managing to a performance metric

o Results are slow
o We’re too agile
“Because no one ever told us to.”
o There are no “performance coverage” requirements
o There isn’t a performance baseline for reference
o There isn’t a defined performance target
o No one is managing to a performance metric
Until Now.

Poll Question
What do you see holding back a more iterative load testing approach? (check all
that apply)
a) Skills gaps
b) Lack of test environment(s)
c) Slow test development
d) Test tools limitations
e) 3rd Party Dependencies
f) Undefined Performance Requirements
g) Lack of Management Push
h) Other

Load Testing Has Been an Elite Event
SThe Issue:
• High expectations from tests run with diminishing time windows
• Code-level performance bugs found late set entire project back
• Separate team and schedule conflict with delivery goals
• Major issues escape to production
Iteration 1
Automated unit
tests
Iteration 2
Automated unit &
functional tests
Iteration 3
Regression test
suite
Iteration n
Regression test
suite
Load &
Performance
Testing
Deploy
Issues?

Continuous Load Testing
Validate performance piece by piece
Iteration 1
Automated unit
tests
Deploy
Production perf
tests
Iteration 2
Automated unit &
functional tests
Iteration 3
Regression test
suite
Incl:
Iteration n
Regression test
suite
Incl:
End to End
Production Scale
Test
Small perf test Small perf tests
Perf smoke tests Perf smoke tests
The Solution:
• Run incremental performance tests automatically with each iteration
• Establish a trend line for performance early
• Developers find code and architecture issues early
• Performance teams focus on end-to-end infrastructure and user related testing
• Minimize production escapes and implement production testing
Development/Test team Performance/Ops teamDevOps

Continuous Performance (for Web & Mobile)
The Path to a Performance Culture
CloudTest runs load
tests via JenkinsDaily (or more frequent) updates keep
teams “performance focused”
Identify
bottlenecks &
inform teams,
just like unit
and functional
tests in
Jenkins
Jenkins launches
suitable cloud-
based test
environment
Manage with a new metric!

Poll Question
What types of Load Testing are you doing today? (Single Answer)
a) Non-Web Protocols (not mobile)
b) Primarily web protocols
c) Primarily mobile web traffic
d) Both web and mobile
e) All web, mobile & non-web protocols

16© 2015 SOASTA, Inc. All rights reserved.
Enabling Continuous Performance
Testing with a Jenkins CI/CD Pipeline
Brian Dawson
Twitter: @brianvdawson
Email: bdawson@cloudbees.com

Why are we here today?
Fundamentally we are talking about......
17
Delivering
Better
Software
Faster!
Concept-to-Customer
High Quality
• Performance
• Feature
• Function
Rapidly
Eliminating Waste
Market Innovation
High Value

How Do You Deliver Better Software Faster?
18
Prod
Rapidly
Repeatedly
Reliably
ProdDev

19
Rapidly
Repeatedly
Reliably
ProdDev

20
“DevOps” Has Great Promise
“DevOps is a software
development method that stresses
communication, collaboration and
integration between software
developers and information
technology (IT) professionals.”[1]
[1] http://en.wikipedia.org/wiki/DevOps
DevOps

21
“DevOps” Has Great Promise
“DevOps is a software
development method that stresses
communication, collaboration and
integration between software
developers and information
technology (IT) professionals.”[1]
[1] http://en.wikipedia.org/wiki/DevOps
DevOps
But You Can’t “Do” DevOps. DevOps an outcome.

You Can “Do” Continuous Delivery
The application of automation and orchestration technologies and
principles of Lean Manufacturing to the software delivery lifecycle.
22

23
DevOpsThe “What”

26
DevOps
The “How”
The “What”
Continuous
Integration
Agile
Iteration
Continuous
Delivery
SDLC Coverage
Feature, Non-Functional, Function, Operational
Integrationand
Automation

27
DevOps
The “How”
The “What”
Continuous
Integration
Agile
Iteration
Process
&
Practices
Continuous
Delivery
CI and Continuous Delivery (CD) become the foundation
for a DevOps transformation.

!Robin Thicke “Blurred Lines”
“Today, meaningful integration happens less in source control and more frequently in a
runtime environment… if you want to quickly verify that the work of two developers is not
conflicting, you need to run API tests, as well as some functional tests scenarios. That means
bringing the various services into an integration test environment, and validating runtime
functionality at a pace similar to the classic CI build loop ”
— Steve Smith 2014
Continuous Integration and todays technologies...
The line between Continuous Integration and Continuous Delivery
has blurred

CI, CD and Feedback

30
So, How do you “Do”
Continuous Delivery?

Automation is the Key
31Photo courtesy of Steve Jurvetson via Flickr

Continuous Delivery (CD) Applies Automation to the
Application Lifecycle
ProdDev BuildCommit Test Stage Deploy
Feedback Loop

CD Starts with Continuous Integration (CI):
Automating Application Build and Test
Continuous Integration
ProdDev
Automation
Engine
Developer
Source
Code
Control
System
Compile
/BuildCode
Scan TestsCode
Commit
Results
Report
BuildCommit Test

Jenkins is the #1 Automation Platform Used for CI
Continuous Integration
Developer
Source
Code
Control
System
Compile
/BuildCode
Scan TestsCode
Commit
Results
Report
BuildCommit Test

Jenkins is Also Your Key to Continuous Delivery
Developer
Source
Code
Control
System
Compile
/BuildCode
Scan TestsCode
Commit
Results
Report
BuildCommit Test Stage Deploy
Deploy
Prod
Deploy
Stage
Complex Delivery Pipelines
Delivery of App and Config
Continuous Delivery
ProdDev

Have you met Jenkins?
#1 CI/CD Automation Engine
• Open Source
• Highly Extensible
About 8 years old
Easy to install/use
1000+ plugins
Widely adopted
• 100K+ installations

Over 1000 Jenkins Plugins for Tool Integration
Integrated Development Environments
(IDEs), editors/compilers/debuggers,
code review and collaboration tools.
Source code control
systems and related asset
management tools.
Automated API-based
testing frameworks.
Release
management and
release automation
software
Automated API-based
testing frameworks, static
code analysis tools.
Load Testing Tools
and Security Scanning
technology
Automated and
manual UAT tools
Release management,
release automation, and
change control software.
Environment and Configuration
Management SoftwareDevelop
Commit
Build &
Integrate
Unit Tests
Release
(to Test)
Automated
Regression
Tests
Release
(to Prod)
User
Acceptance
Tests
Load &
Security
Tests

Jenkins is the Hub of the CD Ecosystem
On-premise or
in the cloud
Plug-ins for all
your tools
RunDeployStageTestBuildCommit

Are You Using Today?
41
Chances are someone in your
organization is.

42

43
How Do You Manage Jenkins and Testing at
Enterprise Scale?

About CloudBees
Jenkins founder, Kohsuke Kawaguchi, CloudBees CTO
We optimize Jenkins for the enterprise
• Jenkins Operations Center
• Jenkins OSS Support
• Enterprise-class plugins
• Commit code and support the Jenkins Community
• Jenkins training and consulting
• Jenkins User Conferences (JUC) and newsletters
…we are the Jenkins Enterprise experts!

CloudBees Offerings

CloudBees Jenkins Enterprise
46
Professional support
High availability
Large installations
Security
Optimized utilization
Continuous Delivery

How Can CloudBees Jenkins Enterprise Help with Your CD
Strategy?
47
Workflow Stage View
Provides deep insight into the execution of your workflow stages so
you can detect problems and performance issues
Manage complex “level-of-detail” testing pipelines
Jenkins Workflow
Created by CloudBees to provide Jenkins with a way to define
flexible delivery pipelines that allow for a rich set of new
delivery scenarios
Manage and share “testing-as-code”
Dynamic Environment Creation
Spin up new test/deployment environments automatically as part of
your delivery pipeline
Deployment Tracking and Verfiication
Track the history of your deployments – including who kicked
off a deployment and what code changes were included in it
Create pipelines that leverage verification tests to determine
the quality of a new deployment
Jenkins Analytics
Monitor and analyze performance across all your Jenkins instances
Monitor, measure and manage adoption of testing, CI and CD best
practicesRestart builds, tests, and deployments where they
break
Leverage workflow checkpoints to restart delivery pipelines
without a full (expensive) rebuild
...central source of orchestration to bridge dev, test, and ops alike
And more…

Jenkins Workflow and Workflow Stage View Enable
Full Continuous Delivery Pipelines
Continuous Delivery
ProdDev
Developer
Source
Code
Control
System
Code
Commit

49
CloudBees Jenkins Workflow Stage View
Restartable Checkpoints
Artifact Downloads

Monitoring, Visualization and Analytics
Jenkins Workflow Stage View
Visibility into Pipeline status
Jenkins Analytics
Cross-Team Build Analytics
… among many other features, tailored for enterprise
organizations using Jenkins @ scale

Version
control
systems
Build & CI Systems
SysAdmins
DevOps
Test @ Scale with w/Docker Containers and
Cloud Support
Docker Hub
Docker Engine
Developer
$ docker
Benefits
• Frictionless movement
from dev > test
• Dev + Ops
collaboration in Hub
Test, Stage or
Production

Additional Jenkins Services from CloudBees
52
Jenkins Certified Training
• Delivered via CloudBees partners
Jenkins Consulting
• Delivered via CloudBees Professional Services Team
Jenkins Newsletter
• Community news and expert tips
Jenkins User Conferences
• Learn from other community members

Continuous Performance Testing
o Demo:
• JBoss Ticket Monster Overview
• SOASTA Cloud Test: Setting up a load test up for unattended runs
• SOASTA TouchTest: Setting up a mobile functional test for unattended runs
• Setting up Continuous Performance Testing
• Jenkins Test Data:
• Reviewing Performance Trending Data: Track performance of each build
• Reviewing Performance Pass/Fail Data: Failing
• Reviewing Functional Test Resultsat are we going to show
What are we going to show

The SOASTA Platform
Performance
Analytics
A Continuous Approach: End-to-End Digital Performance Management

SOASTA Jenkins Plug-in
o Open Source: online at jenkins-ci.org
o Automatically available on CloudBees DEV@cloud
o Complete control over CloudTest/TouchTest Automation from within your Jenkins builds:
o Totally integrated into Jenkins environment

Questions?
Ask in Question window

Resources
Contact SOASTA:
www.soasta.com/cloudtest/
info@soasta.com
866.344.8766
Follow us:
twitter.com/SoastaInc
facebook.com/cloudtest
Knowledge Center
• White Papers
• Webinar Recordings
• Case Studies
CloudLink Community
• Support
• Tutorials
• Video
Contact CloudBees
www.cloudbees.com
info@cloudbees.com
1.781.404.5100
Follow us:
twitter.com/cloudbees
facebook.com/CloudBees
www.SOASTA.com www.CloudBees.com
Try it @ developer.cloudbees.com
Documentation
• CloudBees Jenkins Platform
• Workflow in Jenkins
White Papers:
• Making the Leap from Continuous Integration to
Continuous Delivery
• The Business Value of Continuous Delivery

Continuous Performance Testing and
Continuous Delivery with Jenkins
November 12, 2015
Tom Chavez, SOASTA @TomChavez
Brian Dawson, CloudBees

Soasta | CloudBees webinar 11/12/2015

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Soasta | CloudBees webinar 11/12/2015

Similaire à Soasta | CloudBees webinar 11/12/2015 (20)

Plus de SOASTA

Plus de SOASTA (20)

Dernier

Dernier (20)

Soasta | CloudBees webinar 11/12/2015