Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Use Jenkins For Continuous Load Testing And Mobile Test Automation 
Frank Cohen’s presentation at STARWest 2014, Anaheim, ...
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation 
STARWest 2014 
October 15, 2014 
Frank Cohe...
What We Will Talk About 
•Agile Methodology Expects Build, Deploy, Test Automation 
•A Continuous Integration Strategy 
•A...
Agile Methodology in a Process 
7-day Sprint 
Highest Priority 
Backlog of Changes, 
Improvements 
• App does not run on i...
Agile Without CI?
Why Do Continuous Integration? 
•It’s free and easy 
•Your Test Suites Take Longer Than “Overnight” To Run 
•High Risk App...
Why Avoid Continuous Integration 
•You Work In A Small Shop 
•Stakeholders Unable To Participate 
•No One To Manage Projec...
Continuous DevOps 
•Continuous Integration 
• Dashboard To Testing 
•Continuous Deploy 
• Automatic app deployment 
•Conti...
How Do I Get Jenkins? 
•Run on Cloudbees (Fully Managed) 
•Run on AWS, IBM Softlayer (Self Managed) 
•Download and Run Int...
CloudBees Jenkins Configuration
What Is The Measurement of Success 
•Functional Testing Defines End User Experience Success 
• Browser Compatibility, UX L...
A Testing Strategy for You 
•Working With Today’s Apps, Pages, Tools 
• Serverless Architecture means pages that have stat...
How Do I Get App Testing? 
•Ajax App Testing 
• Sahi, Selenium, WebDriver 
• Unit Testing more difficult, easier to mainta...
Continuous DevOps In Practice 
Data Center Switches Testing 
svn, GIT 
TestNode 
4 
TestNode 
5 
APC Controller Oracle 
1 ...
Scale Up To Multiple Teams 
Data Center Switches Testing 
svn, GIT 
TestNode 
4 
TestNode 
5 
APC Controller Oracle 
1 
Te...
Connect To Continuous Integration, Deployment 
Data Center Switches Testing 
svn, GIT 
TestNode 
4 
TestNode 
5 
APC Contr...
Executing Appium Tests from Jenkins 
#Start appium server 
appium_log=appium.log 
appium_cmd="node <REPLACE_WITH_APPIUM_PA...
Why Treat Functional/Performance Tests Differently? 
•Repurposing Benefits 
• Single Test Scripts for All Testing 
• Pivot...
TestScenario Repurposing
Correlation To Back End Load
Backend Correlation
Strategy for Mobile Test Runtime 
•Run Test in Emulators (Mac and Android) 85% 
•Run Test in Real Devices 10% 
•Run Test w...
Become A CI Architect 
•Draw A Line From Business Risk To Your Integration Plan
How To Get Started 
•Attend Scrum Meetings, Ask A Few Questions 
•Ask The Business Manager for an App About Risks 
•Ask th...
Contents of a CI Plan 
•Identify the Parts 
• Beginning-To-End of User Experience, including Data 
• Units of App Function...
Need A CI Test Plan 
Template? 
Ask sales@appvance.com, it’s free
What We Covered Today 
•Agile Methodology Expects Build, Deploy, Test Automation 
•A Continuous Integration Strategy 
•Aja...
Are You Ready? 
•When You Build Modern Apps And Wonder 
“How Risky Is That App Launch Going To Be?” 
•Call Us At (855) 254...
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
Prochain SlideShare
Chargement dans…5
×

STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation

Frank Cohen's session from the STARWest (Software Test Automation) conference, October 2014. Building mobile apps and Ajax apps effectively requires a blended approach to test automation. 

Frank Cohen, CTO/Founder at Appvance, shows how to leverage the best application performance test methodology with test tools to deliver excellent Ajax, iOS and Android apps. 

Learn how to: 
* Effectively choose a mobile testing method for Agile shops 
* Reduce costs by choosing the best tools (Appium, Jenkins, Appvance PerformanceCloud) 
* Scale-up your mobile testing to thousands of test suites automatically 
* The mix-and-match mobile testing methodology for just-in-time performance testing

  • Identifiez-vous pour voir les commentaires

STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation

  1. 1. Use Jenkins For Continuous Load Testing And Mobile Test Automation Frank Cohen’s presentation at STARWest 2014, Anaheim, California, USA October 1, 2014 www.appvance.com Building mobile apps and Ajax apps effectively requires a blended approach to test automation. Frank Cohen, CTO/Founder at Appvance, shows how to leverage the best application performance test methodology with test tools to deliver excellent Ajax, iOS and Android apps. Learn how to: * Effectively choose a mobile testing method for Agile shops * Reduce costs by choosing the best tools (Appium, Jenkins, Appvance PerformanceCloud) * Scale-up your mobile testing to thousands of test suites automatically * The mix-and-match mobile testing methodology for just-in-time performance testing ! fcohen@appvance.com
  2. 2. STARWest: Use Jenkins For Continuous Load Testing And Mobile Test Automation STARWest 2014 October 15, 2014 Frank Cohen, fcohen@appvance.com, (408) 364-5508
  3. 3. What We Will Talk About •Agile Methodology Expects Build, Deploy, Test Automation •A Continuous Integration Strategy •Ajax and Mobile Testing Methodology •Repurposing for Functional and Performance Testing •Becoming A CI Architect
  4. 4. Agile Methodology in a Process 7-day Sprint Highest Priority Backlog of Changes, Improvements • App does not run on iOS • Rendering problems • UX fails at 45 users • Overnight reports needed Funding and Feedback Sprint Review Acceptance ✓ Unit Tests Pass ✓ Functional Tests Pass ✓ Usability Tests Pass ✓ Perf Under Load Pass ✓ Stress Tests Completed ✓ Production Monitor In Place Daily Work Release
  5. 5. Agile Without CI?
  6. 6. Why Do Continuous Integration? •It’s free and easy •Your Test Suites Take Longer Than “Overnight” To Run •High Risk App Launch •Software build/deploy/test plan of record (How do I?) •Easy to go from functional to performance testing •Keeps you productive, while long projects build/deploy/test
  7. 7. Why Avoid Continuous Integration •You Work In A Small Shop •Stakeholders Unable To Participate •No One To Manage Projects on Jenkins •One-shot project (build, launch, forget) •Deploy/Test Requires Human Intervention •Low Risk App Launches
  8. 8. Continuous DevOps •Continuous Integration • Dashboard To Testing •Continuous Deploy • Automatic app deployment •Continuous Testing • Regression on Commit • Repurpose for Load Tests Test CI Runtime Repository Tickets Monitor
  9. 9. How Do I Get Jenkins? •Run on Cloudbees (Fully Managed) •Run on AWS, IBM Softlayer (Self Managed) •Download and Run Internally (Self Managed) •Then, Plug-in Test As A Service Platform
  10. 10. CloudBees Jenkins Configuration
  11. 11. What Is The Measurement of Success •Functional Testing Defines End User Experience Success • Browser Compatibility, UX Latency, Configuration/Integration Problems •Performance Testing Identifies Business Risk To Launch • How Many People Are We Ready To Serve At Any Given Time •Stress Testing Identifies The Experience As The App Fails • International Bank App Showing Other User Details Testing Measures Business Risk Sprint Review Acceptance ✓ Unit Tests Pass ✓ Functional Tests Pass ✓ Usability Tests Pass ✓ Perf Under Load Pass ✓ Stress Tests Completed ✓ Production Monitor In Place
  12. 12. A Testing Strategy for You •Working With Today’s Apps, Pages, Tools • Serverless Architecture means pages that have state, workflow, communication • Real time analytics and immediate fix - at the same time • Extending tools to support new technology •Beginning To End Testing •Scaling Up Your Team’s Efforts • Higher Productivity • Test Management •Cost Effectiveness and Immediate Support
  13. 13. How Do I Get App Testing? •Ajax App Testing • Sahi, Selenium, WebDriver • Unit Testing more difficult, easier to maintain • JMeter does not work in Ajax •Mobile App Testing • Appium for iOS, Android • Unit Testing of REST and SOAP services using soapUI, HAR, JMeter • Others, very limited support and no repurposing to performance tests
  14. 14. Continuous DevOps In Practice Data Center Switches Testing svn, GIT TestNode 4 TestNode 5 APC Controller Oracle 1 Test Scenario 1 DPL, Logging
  15. 15. Scale Up To Multiple Teams Data Center Switches Testing svn, GIT TestNode 4 TestNode 5 APC Controller Oracle 1 Test Scenario 1 DPL, Logging TestNode 2 TestNode 3 APC Controller 2 Test Scenario 2 TestNode 1 Router Team Testing
  16. 16. Connect To Continuous Integration, Deployment Data Center Switches Testing svn, GIT TestNode 4 TestNode 5 APC Controller Oracle 1 Test Scenario 1 DPL, Logging TestNode 2 TestNode 3 APC Controller 2 Test Scenario 2 TestNode 1
  17. 17. Executing Appium Tests from Jenkins #Start appium server appium_log=appium.log appium_cmd="node <REPLACE_WITH_APPIUM_PATH>/appium/lib/server/main.js" echo "[INFO] Appium command: $appium_cmd" $appium_cmd &> $appium_log& appium_pid=$! #check if appium was successfully started sleep 1 appium_pid_start=`ps -p $appium_pid | grep node | awk '{ print $1 }'` if [ -z $appium_pid_start ] ; then echo "[ERROR] Appium failed to start! Port $appiumport may be already in use. " exit 1 fi echo "[INFO] Appium started with pid: $appium_pid . Logging to: ${appium_log}" # be sure to kill appium on script stop trap "kill $appium_pid" SIGINT SIGTERM EXIT A B C
  18. 18. Why Treat Functional/Performance Tests Differently? •Repurposing Benefits • Single Test Scripts for All Testing • Pivot Back and Forth Rapidly • Natural for Continuous Integration and Automation •Performance Test • Run at Low, Medium, High Virtual User Levels • Scalability Index Report Answers Risk Questions
  19. 19. TestScenario Repurposing
  20. 20. Correlation To Back End Load
  21. 21. Backend Correlation
  22. 22. Strategy for Mobile Test Runtime •Run Test in Emulators (Mac and Android) 85% •Run Test in Real Devices 10% •Run Test with Real People 5% We’re going to run the test between 1-2 pm, get as many real people on it then
  23. 23. Become A CI Architect •Draw A Line From Business Risk To Your Integration Plan
  24. 24. How To Get Started •Attend Scrum Meetings, Ask A Few Questions •Ask The Business Manager for an App About Risks •Ask the Developers Where The Test Coverage Is Thin •Choose An Appropriate Test Script Authoring Tool •Write or Record A Test •Commit The Test to svn, CVS, other repository •Plug-in To CI and Maintain
  25. 25. Contents of a CI Plan •Identify the Parts • Beginning-To-End of User Experience, including Data • Units of App Function •Success/Fail Criteria • 50,000 User Transactions per 10 Minutes with less than 1% Errors • Functional Test on IE 8, Firefox 16, iOS 5 •User Archetypes and Workflow •Performance Under Load and Stress
  26. 26. Need A CI Test Plan Template? Ask sales@appvance.com, it’s free
  27. 27. What We Covered Today •Agile Methodology Expects Build, Deploy, Test Automation •A Continuous Integration Strategy •Ajax and Mobile Testing Methodology •Repurposing for Functional and Performance Testing •Becoming A CI Architect
  28. 28. Are You Ready? •When You Build Modern Apps And Wonder “How Risky Is That App Launch Going To Be?” •Call Us At (855) 254-1164 Or Email Us Sales@Appvance.Com And We Will Show You

×