Serena Webcast: Accelerating Application Delivery with Continuous Testing
•Download as PPTX, PDF•
3 likes•964 views
This document summarizes a presentation given by Soasta and Serena Software on accelerating application delivery through continuous testing. The presentation discusses Serena's deployment automation software and how it can increase deployment velocity. It also discusses Soasta's cloud-based testing platform and how it enables continuous testing of applications from development through production. Speakers from both companies provide examples of how their tools have helped customers reduce testing cycles and speed up time to market for applications. The presentation emphasizes the benefits of a continuous testing approach over a traditional linear development and testing model.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (11)
Similar to Serena Webcast: Accelerating Application Delivery with Continuous Testing
Similar to Serena Webcast: Accelerating Application Delivery with Continuous Testing (20)
More from Serena Software
More from Serena Software (20)
Recently uploaded
Recently uploaded (20)
Serena Webcast: Accelerating Application Delivery with Continuous Testing
- 1. Accelerating Application Delivery with Continuous Testing Presented by Soasta and Serena Software November 20, 2014 ©2014 CloudBees, Inc. All Rights Reserved
- 2. Introduction to Serena • Founded 1980 • Headquartered in San Mateo, CA 2 • >$200M revenue • 200+ R&D Engineers • HGGC Portfolio Company • Follow the sun Support • Serena Deployment Automation • Increase Deployment Velocity • Quality, Predictability, Repeatability • Visual Process Definition • Helping Global 2000 enterprises build and release better software
- 3. Today’s speakers Today’s speakers 3 Mark Levy DevOps Evangelist, Serena Peter Galvin SVP Marketing, SOASTA
- 4. Expertise © 2014 SOASTA. All rights reserved. 4
- 5. GARTNER Magic Quadrant for Integrated Software Quality Suites August 2014 © 2014 SOASTA. All rights reserved. 5 COMPLETENESS OF VISION ABILITY TO EXECUTE CHALLENGERS LEADERS NICHE PLAYERS VISIONARIES “SOASTA has consistently provided innovation for mobile and web applications”
- 6. Results Comparison Jan 2014 6 SERENA SOFTWARE INC. Oct 2014
- 7. Rapid Delivery Agile development Continuous delivery DevOps principles © 2014 SOASTA. All rights reserved. 7 Mobile and Web Online everything Mobile commerce Legacy modernization Complexity Composite apps 3rd party services API proliferation Unprecedented Scale 100’s to millions Local to global 100% availability
- 8. © 2014 SOASTA. All rights reserved. 8
- 9. © 2014 SOASTA. All rights reserved. 9 o Small Batch Sizes
- 10. © 2014 SOASTA. All rights reserved. 10 o Small Batch Sizes o Comprehensive Version Control
- 11. © 2014 SOASTA. All rights reserved. 11 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies
- 12. © 2014 SOASTA. All rights reserved. 12 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation
- 13. © 2014 SOASTA. All rights reserved. 13 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback
- 14. © 2014 SOASTA. All rights reserved. 14 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds
- 15. © 2014 SOASTA. All rights reserved. 15 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds o Consistent environments
- 16. © 2014 SOASTA. All rights reserved. 16 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds o Consistent environments o Developer – Tester Collaboration
- 17. 50,000 Apps 1,000,000 Apps 1,500,000 Apps © 2014 SOASTA. All rights reserved. 17
- 18. © 2014 SOASTA. All rights reserved. 18
- 19. © 2014 SOASTA. All rights reserved. 19 “We didn’t have enough testing for a very complicated project” US Health and Human Services Secretary, Kathleen Sebelius
- 20. © 2014 SOASTA. All rights reserved. 20
- 21. © 2014 SOASTA. All rights reserved. 21 Mobile Test Automation
- 22. © 2014 SOASTA. All rights reserved. 22 Mobile Test Automation Real Test Devices
- 23. © 2014 SOASTA. All rights reserved. 23 Mobile Test Automation Performance Testing Real Test Devices
- 24. © 2014 SOASTA. All rights reserved. 24 Mobile Test Automation Performance Testing Real Test Devices Solid Integration
- 25. Does it work as designed? Assembly Line Approve materials Real Drivers Does it perform under extreme circumstances? © 2014 SOASTA. All rights reserved. 25 What happens in actual conditions?
- 26. Functional validation Build Real Users Performance, reliability, scalability tests © 2014 SOASTA. All rights reserved. 26 Unit-tested code What happens in actual conditions?
- 27. © 2014 SOASTA. All rights reserved. 27 o Rapid & accurate test development o Reliable, maintainable test automation o Stability across mobile OS platforms o Device performance measurement o Real-Time feedback via Continuous Integration
- 28. o Fully managed device cloud installed behind the firewall o Real-time device inventory, status and availability o Online access for all your teams o 24x7, secure, remote access to devices o Manual & Automated functional test o Remotely access to selected mobile device and any mobile app • Use devices for manual testing and monitoring • Remote screen-share devices for collaboration on design or issues © 2014 SOASTA. All rights reserved. 28
- 29. o Dedicated Public Device Cloud o Access to the latest market leading devices (iOS/Android) o Automatic installation, maintenance and upgrade o Hosted devices in a secure data center o Stock devices, not jail broken/rooted o Seamless integration with TouchTest and CI environments o API available to install apps, reboot/wake-up devices, launch tests Perfect for 24x7 Automated Mobile Regression Testing © 2014 SOASTA. All rights reserved. 29
- 30. Load Testing at All Stages Performance validation from development to production Continuous tuning for full-scale…and well beyond s Latency between systems Network bandwidth Search technology limits © 2014 SOASTA. All rights reserved. 30 Memory leaks Test Scale Stage / Team Network & Ops Release & Deploy Dev & Test CDN file placement Load Balancer configuration Network configuration DNS routing Inadequate server resources Default configuration settings Auto-scaling failures Unbalanced web servers Slow third-party plug-ins Garbage collection Database thread counts Inefficient database queries Slow pages Conflict with other apps Test Lab Staging Production (100% +++) Method-level tuning Max sockets exceeded Firewall max capacity Global latency variance Security bottlenecks
- 31. Load Testing at All Stages Performance validation from development to production Continuous tuning for full-scale…and well beyond s Latency between systems Network bandwidth Search technology limits © 2014 SOASTA. All rights reserved. 31 Memory leaks Test Scale Stage / Team Network & Ops Release & Deploy Dev & Test CDN file placement Load Balancer configuration Network configuration DNS routing Inadequate server resources Default configuration settings Auto-scaling failures Unbalanced web servers Slow third-party plug-ins Garbage collection Database thread counts Inefficient database queries Slow pages Conflict with other apps Test Lab Staging Production (100% +++) Method-level tuning Max sockets exceeded Firewall max capacity Global latency variance Security bottlenecks
- 32. Load Testing at All Stages Performance validation from development to production Continuous tuning for full-scale…and well beyond s Latency between systems Network bandwidth Search technology limits © 2014 SOASTA. All rights reserved. 32 Memory leaks Test Scale Stage / Team Network & Ops Release & Deploy Dev & Test CDN file placement Load Balancer configuration Network configuration DNS routing Inadequate server resources Default configuration settings Auto-scaling failures Unbalanced web servers Slow third-party plug-ins Garbage collection Database thread counts Inefficient database queries Slow pages Conflict with other apps Test Lab Staging Production (100% +++) Method-level tuning Max sockets exceeded Firewall max capacity Global latency variance Security bottlenecks
- 33. Load Testing at All Stages Performance validation from development to production Continuous tuning for full-scale…and well beyond s Latency between systems Network bandwidth Search technology limits © 2014 SOASTA. All rights reserved. 33 Memory leaks Test Scale Stage / Team Network & Ops Release & Deploy Dev & Test CDN file placement Load Balancer configuration Network configuration DNS routing Inadequate server resources Default configuration settings Auto-scaling failures Unbalanced web servers Slow third-party plug-ins Garbage collection Database thread counts Inefficient database queries Slow pages Conflict with other apps Test Lab Staging Production (100% +++) Method-level tuning Max sockets exceeded Firewall max capacity Global latency variance Security bottlenecks
- 34. CDN Web App Server Server Web Server Web Server Web Server App Server App Server App Server Load Balancer Cache Cache Databas e © 2014 SOASTA. All rights reserved. 34 CloudTest Analytics Main Databas e Analytics Load GeneLroaatodr GeneraLtooar d Perf Tester 1 Generator Analytics San Francisco (GoGrid) Load GeneLroaatodr GeneraLtooar d Generator Tokyo (AWS) Analytics Load GeneLroaatodr Generator Amsterdam (Azure) Data sources: Native (SSH), JMX, PerfMon, CA Introscope, AppDynamics, New Relic, CloudWatch,…. Analytics Load GeneLroaatodr Generator Virginia (IBM) Analytics Load GeneLroaatodr GeneraLtooar d Generator SOASTA Global Test Cloud Access for all Analytics Load GeneLroaatodr GeneraLtooar d Generator East Coast (AWS) “Conductor” Credit Card Web analytics Other 3rd pty svcs Chicago (Rackspace)
- 35. © 2014 SOASTA. All rights reserved. 35
- 36. Jenkins runs any unit tests as usual © 2014 SOASTA. All rights reserved. 36
- 37. Spin up suitable, consistent test environments in the cloud or the lab Run iterative load tests via Jenkins Jenkins runs any unit tests as usual © 2014 SOASTA. All rights reserved. 37
- 38. Spin up suitable, consistent test environments in the cloud or the lab Run iterative load tests via Jenkins Mobile? Run functional automation on real devices for functional & performance validation Jenkins runs any unit tests as usual © 2014 SOASTA. All rights reserved. 38
- 39. Spin up suitable, consistent test environments in the cloud or the lab Run iterative load tests via Jenkins © 2014 SOASTA. All rights reserved. 39 Daily (or more frequent) updates keep teams quality focused Embedded failure details & bottleneck data isolate issues Mobile? Run functional automation on real devices for functional & performance validation Jenkins runs any unit tests as usual
- 40. Spin up suitable, consistent test environments in the cloud or the lab Run iterative load tests via Jenkins © 2014 SOASTA. All rights reserved. 40 Daily (or more frequent) updates keep teams quality focused Manage to a performance baseline Embedded failure details & bottleneck data isolate issues Mobile? Run functional automation on real devices for functional & performance validation Jenkins runs any unit tests as usual
- 41. Spin up suitable, consistent test environments in the cloud or the lab © 2014 SOASTA. All rights reserved. 41 Use real user data to iterate dev & test plans Run iterative load tests via Jenkins Daily (or more frequent) updates keep teams quality focused Manage to a performance baseline Embedded failure details & bottleneck data isolate issues Mobile? Run functional automation on real devices for functional & performance validation Jenkins runs any unit tests as usual
- 42. © 2014 SOASTA. All rights reserved. 42
- 43. © 2014 SOASTA. All rights reserved. 43
- 44. Traditional Linear Approach Dev Ops © 2014 SOASTA. CONFIDENTIAL – Not for Distribution All rights reserved. 44 Ut Unit Testing Pe Performance Testing Fu Functional Testing Development Testing Operations TIME Test Continuous Testing Approach
- 45. © 2014 SOASTA. All rights reserved. 45 SOASTA Customers “4 months of manual tests now run in 1 hour, overnight.” “25% reduction in time to market for production rollouts.” Steven Winter, Director of Quality FIS Mobile – Mobile banking platform to over 1000 banks
- 46. o 250 hours of manual tests running in just over 1 hour every night o Regression Tests not run until feature complete now run nightly – 4 months to overnight o Fixing bugs earlier – Detailed test results for developers o 25% Reduction time to market © 2014 SOASTA. All rights reserved. 46 o Reduced field failures
- 47. “We decreased our test cycles from three weeks to a couple of days.” © 2014 SOASTA. All rights reserved. 47 SOASTA Customers “TouchTest has been critical in improving both the speed to deliver and the quality of our mobile apps.” Thierry Chassaing, SVP of Product & Engineering. ItsOn - the leader in Mobile Smart Services.
- 48. © 2014 SOASTA. All rights reserved. 48 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 49. © 2014 SOASTA. All rights reserved. 49 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 50. © 2014 SOASTA. All rights reserved. 50 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 51. © 2014 SOASTA. All rights reserved. 51 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 52. © 2014 SOASTA. All rights reserved. 52 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 53. Questions? Upcoming DevOps Drive-In 53 Please use the Q&A panel to submit your questions Free Deployment Automation December 18th CD: Integrating the Deployment Friday 1 pm ET/ 10 am PT Pipeline Toolchain through Automation Weekly Deployment Automation Demo Get the Free Version
- 54. Want to See More? 54 serena.com/sda
- 55. Thank you!
Editor's Notes
- Greeting everyone this is Mark Levy, product marketing manager here at Serena and I will be your host today. Serena Software help IT organizations build and release better software. Head over to serena.com to find out more about Serena’s products and solutions. Check out Serena Deployment Automation Community Edition.
- This customer success and consistent innovation has resulted in SOASTA’s recognized market leadership by Gartner. The legacy leaders are not keeping up with modern demands. Everyday, we are replacing the decades-old tools with technology and services build for this century…for this decade. As the youngest and only private company in the leaders category, we never slow down in our pursuit of innovation and customer satisfaction.
- http://www.forbes.com/sites/theapothecary/2013/10/23/now-she-tells-us-sebelius-says-obamacares-exchange-website-needed-six-years-of-development-instead-of-two/ Millions of uninsured citizens Healthcare program mistrust President’s approval rating dropped Congress tied up in legislation Public cynicism Press and social media circus Uncertainty for the future
- Rapid and accurate test development – All gestures, pinch, zoom and now shake Reliable replay Record & replay with velocity & motion Built in validations In-app control Measure impact of the app on the device – CPU, Battery, Memory, signal strength SOASTA’s new approach embeds a small library inside the object repository of the application allowing the developer to have complete control over the test process. In particular this approach lends itself to a much more stable Test Platform that captures with precision all forms of movement associated to the use of gestures. It also enables a mobile developer to test applications without jail breaking a device. This is a huge issue today in the iOS environment. SOASTA TouchTest as an example I “ still” the only test automation platform that can test an app on the iPad mini. This new approach invented by SOASTA’s Advanced Laboratories we believe will become the standard approach to testing mobile apps in the future.
- All in all, there can be several environments that you have to manage or that your team has to be aware of, and your teams are probably distributed across the world and from all different timezones. So with that baseline, it’s important to performance test your application from end-to-end; starting from your test lab to production. As you view this chart, you can see the potential issues that can arise in your application. Real problems like 3rd party services, CDNs, bandwidth will only manifest itself in production; typically not in your test lab or your staging environment. By building a test plan that includes production, you are more likely to discover issues that can only show up in production - an environment that matters the most for your company and customers. An environment that drives business value, revenue, and brand awareness and impressions.
- All in all, there can be several environments that you have to manage or that your team has to be aware of, and your teams are probably distributed across the world and from all different timezones. So with that baseline, it’s important to performance test your application from end-to-end; starting from your test lab to production. As you view this chart, you can see the potential issues that can arise in your application. Real problems like 3rd party services, CDNs, bandwidth will only manifest itself in production; typically not in your test lab or your staging environment. By building a test plan that includes production, you are more likely to discover issues that can only show up in production - an environment that matters the most for your company and customers. An environment that drives business value, revenue, and brand awareness and impressions.
- All in all, there can be several environments that you have to manage or that your team has to be aware of, and your teams are probably distributed across the world and from all different timezones. So with that baseline, it’s important to performance test your application from end-to-end; starting from your test lab to production. As you view this chart, you can see the potential issues that can arise in your application. Real problems like 3rd party services, CDNs, bandwidth will only manifest itself in production; typically not in your test lab or your staging environment. By building a test plan that includes production, you are more likely to discover issues that can only show up in production - an environment that matters the most for your company and customers. An environment that drives business value, revenue, and brand awareness and impressions.
- All in all, there can be several environments that you have to manage or that your team has to be aware of, and your teams are probably distributed across the world and from all different timezones. So with that baseline, it’s important to performance test your application from end-to-end; starting from your test lab to production. As you view this chart, you can see the potential issues that can arise in your application. Real problems like 3rd party services, CDNs, bandwidth will only manifest itself in production; typically not in your test lab or your staging environment. By building a test plan that includes production, you are more likely to discover issues that can only show up in production - an environment that matters the most for your company and customers. An environment that drives business value, revenue, and brand awareness and impressions.
- Key Messages: The Market: Mobile App Performance (Predictive, Contextual, and Actionable Analytics) Market Size: $10B (10M Apps, 3.5M Users, Trillions of User Experiences, $200B Mobile economy by 2017) Why SOASTA is the The Winner in Mobile Performance Technology: Leadership (Cloud, Performance Analytics, Test Automation) - Patents People: Clear Leadership (Yahoo, Walmart, Intuit, Experiean, Mercury, Gomez, Keynote) Customers: Enterprise and Global Global Customers: Offices, Customers Competition: NewRelic, AppDynamics,and PerfectoMobile Partnership: Dev (Appcellerator-OEM), Deploy (AWS & 17 others), Monitor (NewRelic and 15 others) Business Model: SaaS Customers Deal Size: $5K - $2.5M TCV Focus on $100M in Sales by 2015
- FIS World’s largest banking and Payments provider Nation’s largest Mobile Banking and Payments provider +1,200 banks, +25MM end-users Most successful mPayments app Most successful mBanking app Award winning Mobile Wallet CI is absolutely necessary to compete in time-to-market and scalability CI in mobile is a big paradigm shift Faster and more specific test results to developers so they can work smarter Identify integration issues sooner Enables Continual Quality Paychex Delivering Payroll, Time and Labor, 401K and Benefits packages for over 500,000 small and medium businesses is resource intensive. Setting an ambitious goal in 2011 to grow their business by 50% over three years, Paychex knew that achieving it would require serving existing customers at a lower cost, reducing the cost of customer acquisition, and finding new ways to grow – that meant moving applications to the web and creating a SaaS model. Doing that meant changing the way they approached application development, quality, and customer experience. They turned to SOASTA to enable them to test internally and externally, test applications concurrently, reduce defect rates and test earlier in the life-cycle, improve mobile quality, and understand user experience.
- Paychex Delivering Payroll, Time and Labor, 401K and Benefits packages for over 500,000 small and medium businesses is resource intensive. Setting an ambitious goal in 2011 to grow their business by 50% over three years, Paychex knew that achieving it would require serving existing customers at a lower cost, reducing the cost of customer acquisition, and finding new ways to grow – that meant moving applications to the web and creating a SaaS model. Doing that meant changing the way they approached application development, quality, and customer experience. They turned to SOASTA to enable them to test internally and externally, test applications concurrently, reduce defect rates and test earlier in the life-cycle, improve mobile quality, and understand user experience.