How to use Jenkins for Continuous Load Testing and Mobile Automation
Today’s rapid development pace demands continuous testing, and Jenkins, the leading open source automation platform, has emerged as the hub of continuous delivery. SOASTA and CloudBees have tapped Jenkins to enable more test types and approaches that utilize cloud and agile processes for higher quality apps.
Join this free webinar and learn:
How to use Jenkins for continuous delivery and load testing of mobile applications
How to incorporate cloud resources into your development and test environments
Using the largest global test cloud for load generation
CloudBees’ on premise, in the cloud and hybrid solutions for continuous delivery with Jenkins
SOASTA’s Jenkins plugins for testing with real mobile devices and tracking performance baselines
Experts from both companies will share stories and demonstrations that will help you implement a continuous approach to quality.
2. 2
CI Gets More Testing
TODAY’S PRESENTERS
Andy Pemberton: CloudBees, Sr. Dir. Solution Architecture
Mike Ostenberg: SOASTA, Dir. Solutions Engineering
Brad Johnson: SOASTA, VP Product
TWO COMPANIES, A SINGLE SOLUTION
- the leader in Mobile and Cloud Testing
- the Jenkins Enterprise company
Together, we offer a complete cloud-based service:
Continuous Delivery for Web & Mobile
@apemberton
@bradjohnsonsv
3. 3
• Achieving (even) more test coverage with Jenkins
• Utilizing cloud in development and test
• CloudBees and Jenkins
• From Continuous Integration to Continuous Delivery
• CI with SOASTA CloudTest (load) and TouchTest (mobile)
Questions:
Submit in the question box during event
4. 4
Testing
The Legacy of….WAIT
Ut
Unit Testing
Ft
Functional
Testing
Development Testing Operations
Pt
Performance
Testing
Mo
Monitoring
Legacy Approach: Silos and waiting
TIME
5. 5
Eliminating Delays Compresses Time
Enabling more testing to be done earlier and more often
Ut
Unit Testing
Ft
Functional
Testing
Testing
Pt
Performance
Testing
Mo
Monitoring
Legacy Approach: Silos and waiting
Agile Approach: Automation and Continuous Testing
OperationsDevelopment
Dev Ops
Testing
TIME
6. 6
* Source: Gartner, “Blueprint for Adding Continuous Integration as a Start to Your Continuous Delivery Process” Sean Kenefick, Aug 25, 2014
Traditionally
later-stage
testing!
Perf & mobile
testing plugins
enhance value
across delivery
cycle
9. o #1 OSS CI server
o Written in Java
o Highly Extensible
o Founded ~2005
o Easy to install / use
o Extensible via 1000+ plugins
o Widely adopted
o 80K+ installations
o CloudBees knows Jenkins
o On-premise, cloud and hybrid
o Expert support
o Enterprise plugins
Have you met Jenkins?
10. o Jenkins founder, Kohsuke Kawaguchi, CloudBees CTO
o We optimize Jenkins for the enterprise
o Jenkins Operations Center
o Jenkins OSS Support
o Enterprise-class plugins
o Commit code and support the Jenkins Community
o Jenkins training and consulting
o Jenkins User Conferences (JUC) and newsletters
o …we are the Jenkins Enterprise experts!
CloudBees
11. CloudBees Jenkins Products
On-Premise In the Cloud Hybrid
No matter how you use Jenkins
Jenkins Enterprise
by CloudBees
Jenkins Operations Center
by CloudBees
DEV@cloud DEV@cloud
Hybrid
12. Jenkins: the Hub of Continuous Delivery
Source Code
Binaries
Quality Analysis Functional Test
Performance / Load Test
Production
13. Jenkins: the Hub of Continuous Delivery
Source Code
Binaries
Quality Analysis Functional Test
Performance / Load Test
Production
14. o “Survival is not mandatory
(nor is adopting DevOps)”
o Gene Kim, Wall Street Journal
- CIO Journal DevOps piece
o Seek out opportunities for
automation - testing is an
ideal first step
o Load, functional, and
mobile testing have been
challenging…
DevOps is a Journey
http://blogs.wsj.com/cio/2014/05/22/enterprise-devops-adoption-isnt-mandatory-but-neither-is-
survival/
8x more
frequent
releases
8000x faster
release process
2x higher
releases
success
12x faster
defect fixes
DevOps
16. 16
o Demo:
• Creating a quick functional test in TouchTest
• Creating a quick performance test in CloudTest
• Running Load and Performance Tests from Jenkins:
• Play TouchTest/CloudTest tests from Jenkins
• Setting Thresholds to automatically alert on poor performance
• Plotting performance trending data directly in Jenkins
• Reviewing detailed results directly in Jenkins
• Jenkins Enterprise by CloudBees enhancements
• Add templates
• Add folders
• Add roles and permissions
18. 18
TouchTest
o Run MATT utility to create
testable application
o Install Testable application
on device
CloudTest
o Install conductor on laptop
o Modify mobile device proxy
settings
Jenkins
o Install SOASTA PlugIn
o Install Plot PlugIn
o Install Jenkins Enterprise by
CloudBees
19. • Open Source: online at jenkins-ci.org
• Automatically available on CloudBees DEV@cloud
• Complete control over CloudTest/TouchTest Automation from within your
Jenkins builds:
• Totally integrated into Jenkins environment
SOASTA Jenkins Plug-in
23. 23
Thanks & Q&A
Contact SOASTA:
www.soasta.com/cloudtest/
info@soasta.com
866.344.8766
Follow us:
twitter.com/cloudtest
facebook.com/cloudtest
Knowledge Center
• White Papers
• Webinar Recordings
• Case Studies
CloudLink Community
• Support
• Tutorials
• Video
RESOURCES
Contact CloudBees
www.cloudbees.com
info@cloudbees.com
1.781.404.5100
Follow us:
twitter.com/cloudbees
facebook.com/CloudBees
www.SOASTA.com www.CloudBees.com/jenkins
blog.cloudbees.com
developer.cloudbees.com
• Platform Services
• Frameworks
Notes de l'éditeur
So, based on the poll results…. (there is a mix of users that test in development, staging, and production environment) …
In today’s webinar, we will be focusing on performance testing as a continuous process, testing across your software product lifecycle - devops and perf eng is a team effort! Load testing has evolved along with continuous delivery and now requires that developers eliminate performance issues on their desktop while test and operations teams validate from the test lab to live production systems. Your test platform must be adaptable and accessible testing behind and beyond firewalls.
Generally speaking, traditionally, testing has begun at the component level and works outward toward the integration of the entire system. At each point of this process, there are different techniques to testing; however, with the market being so competitive and your users/customers expecting so much out of your product, the time to get this important function done may be compromised.
Unfortunately, testing is often the bottleneck that causes teams to wait as manual testing completes or redundant testing tasks are executed by siloed test teams.
The goal is not to introduce delays but instead to continually test throughout the lifecycle.
While enabling more testing at all stages of the product lifecycle, we want to eliminate delays and compress the time to release. This approach requires communication and collaboration between various stakeholders - dev folks, QA team, the product owner and the Ops team. Whether your web app is for external or internal users, the performance of the app is critical – they want a fast and responsive site to get their job done. Most of us have been through this … but do we have the patience to sit and watch the spinning circle and see the pages render slowly down the browser? Perceived wait time can causes positive or negative reactions to your product.
In an agile approach, automation and continuous testing compress time.
Developers test in short iterations on their local machines and conduct unit tests. A big change we see in more agile shops using SOASTA is that the Dev team is empowered to build automated functional and performance tests on their code. Performance has traditionally been the domain of specialized teams, and still is, but if Developers eliminate code-level performance bugs, the Perf Team can build better and more complex tests to isolate issues outside of the code. Test and Ops team can then focus on tests with different objectives behind existing firewalls … The ops team can also do the same approach, working in conjunction with the developers. But the difference here is that the testing gets expanded focusing on areas of the code that involve your infrastructure – database, APIs / web services, smoke tests, custom code, critical single page apps.
This bottom-up testing scenario gives tests low level components combined in clusters to validate the performance of a specific software function with increasingly complex and user-focused testing as time goes on.
Work very closely with the community – lead sponsor of Jenkins User Conferences
Focus is the plugin architecture
High availability, scalability
Jenkins is emerging as the hub of continuous delivery
Now let’s dig a little deeper into SOASTA Solutions. Running on our single platform, each set of capabilities are packaged to provide a complete set of quality tools for your teams to meet the needs of an increasingly continuous delivery cycle.