Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Continuous Performance Testing and
Monitoring in Agile Development
Deliver faster applications faster through continuous performance validation
Who We Are
Who We Are
Mission: Deliver faster applications faster through continuous performance validation
Who We Are
Mission: Deliver faster applications faster through continuous performance validation
User Demands Process Revolutions
We’re Driven by Your Challenges
Technology Transitions
AGENDA
Performance matters
Agile testing
From Agile to DevOps
DevOps pipeline
Dynatrace Integration
40% of users are moving towards the
competition following a bad experience
57 % of users use
their mobile
44-61 % of users...
Usability
Ergonomic
Performance
Accessibility
Features
What is User Experience?
Etam reduces it's average page load time from 1.2s to 500ms and increased
conversions by 20%, time on site by 21%, and pag...
AGENDA
Performance matters
Agile testing
From Agile to DevOps
DevOps pipeline
Dynatrace Integration
Performance Testing Used To Be ….
Performance Design is The Key
Standard Performance Testing Workflow
Strategy
Scenarios
Prepare
Execute
AnalyzeTuning
Testing Earlier
Early Risk analysis
• Understand the application
• Performance needs to be included in the
PRA
• ATAM ( Ar...
Performance is driven by :
• The end users
• Third party system
• …
We need to ask the right questions :
• How the users w...
Performance testing an existing application
• Ask the support of functional architect, HR..etc
• Understand the behavior o...
Unit Performance Testing
Any part of the system
Not a standard practice
Do not wait until the system is assembled
Test cas...
Change Mentality
Before
 Late record/playback
performance testing
 System-level requirements
 Record/playback approach
...
Workflow
3) Development life cycle
Component
Testing
Business
Test Case
1) Specification
2) Risk Analysis
Project Lifecycle
Neotys Continuous Performance Validation Solution
Component
testing
Requirements
Daily Cycle
Performance
End-to-End testin...
AGENDA
Performance matters
Agile testing
From Agile to DevOps
DevOps pipeline
Dynatrace Integration
Organizations Adopt Agile and DevOps as an Answer
• 99% of organizations have adopted Agile
development methods
• 88% of C...
But Speed is Nothing Without Quality
“Surviving Disruption, Leading Change:
Winning in the Application Economy,” 2015
2/3 ...
“The term everyone is using is
DevOps, but I think it should
really be DevTestOps. Testing
is really crucial for achieving...
AGENDA
Performance matters
Agile testing
From Agile to DevOps
DevOps pipeline
Dynatrace integration
Planning
• Provide transparency to the stakeholders
Source Control
• Tracking Change of the code ( infrastructure, applica...
Stage 1: Build
Build
request
Prepare
Build
Compile
Packaging
Stage 2: Deploy
Deploy
DEV QA UAT
Stage 3: Test
receive task
integration
test
DEV QA UAT
Test
Integration Functional Performance Security
Stage 4 : Release
Release
PROD
receive task
integration
test
Devops toolchain
Build
request
Deploy Release
DEV QA UAT
PROD
Test
Prepare
Build
Compile
Pac...
AGENDA
Performance matters
Agile testing
From Agile to DevOps
DevOps pipeline
Dynatrace integration
• response time
• infrastructure behavior
Capture performance metrics that go beyond
• Failures, bottlenecks, slower-than-...
How Does the Integration Works?
Web Model
Mobile
Model
Rest
Interface
Business
logic
Data Access
logic
XML
Legacy
Relation...
• Transaction name, virtual user, script name, unique id
• Easier correlation between load testing solution and Dynatrace ...
• Download our free version of NeoLoad including the integration with
Dynatrace
Start LoadTesting with NeoLoad now
• Conta...
Next Steps
henrik.rexed@neotys.com
Visit Neotys Community at answers.neotys.com
Download NeoLoad Free Edition from www.neo...
mobile
browser
network
multi-geo
3rd parties cloud
containers
services
code
hosts
synthetic
logs
business
transaction
appl...
Synthetic Agents
Deployed and managed
by Dynatrace
Backbone Last Mile
Web Performance
Management
100+ locations
Web Perfor...
Database and Mainframe
1st Tier
N-Tier
Front End 0.2s
App Tier 1.1s
Middleware 0.3s
Database 0.1s
Login
Search
Order Stock...
Inspect single Users
Biz/App
User Experience Management
Deep Transaction Analysis
It‘s not about blind automation of pushing more
bad code through a shiny pipeline
It‘s not about blindly giving everyone Ops power
to deploy changes only tested locally
Level-Up your Functional Tests with Metrics
Build 17 testPurchase OK
testSearch OK
Build # Test Case Status
Test & Monitor...
Level-Up your Functional Tests with Metrics
Build 17 testPurchase OK
testSearch OK
Build 18 testPurchase FAILED
testSearch...
Level-Up your Functional Tests with Metrics
Build 17 testPurchase OK
testSearch OK
Build 18 testPurchase FAILED
testSearch...
Level-Up your Functional Tests with Metrics
Build 20 testPurchase OK
testSearch OK
Build 17 testPurchase OK
testSearch OK
...
Level-Up your Functional Tests with Metrics
Build 20 testPurchase OK
testSearch OK
Build 17 testPurchase OK
testSearch OK
...
Level-Up your Functional Tests with Metrics
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPurcha...
Level-Up your Functional Tests with Metrics
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPurcha...
Level-Up your Functional Tests with Metrics
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPurcha...
#1: Analyzing every Unit,
Integration & REST API test
#2: Key Architectural
Metrics for each test
#3: Detecting regression...
Quality Overview by Build
In Dynatrace …
#8: Integrate into your Delivery Pipeline
CI/CD
Quality Overview by Build
In Dynatrace …
… allows you to Stop a Bad Build
in Jenkins, NeoLoad, …
#8: Integrate into your D...
One goal: deliver better features to customers faster
Two fundamental components: speed + quality
Take the next step with a Dynatrace Free Trial !!!
dynatrace.com/trial
Sign up at:
Confidential, Dynatrace, LLC
Q & A
Henrik Rexed
Performance Engineer
@hrexed
Asad Ali
Product Specialist
@AsadThoughts
Dow...
Continuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile Development
Continuous Performance Testing and Monitoring in Agile Development
Prochain SlideShare
Chargement dans…5
×

Continuous Performance Testing and Monitoring in Agile Development

1 338 vues

Publié le

Continuous Performance Testing and Monitoring in Agile Development

Continuous Performance testing and monitoring is the best way to ensure application performance with quicker development cycles. Balancing agile and DevOps velocity with the need for ongoing performance testing and monitoring is essential. We call it Continuous Performance Validation.

In this webinar, we will show how you can get performance guidance and metrics throughout development, making sure apps perform well from inception to production and beyond.

In this webinar you will learn:
• How to automate performance testing and which tools you need to be successful
• How to use APM during load and performance testing
• How to create a continuous performance validation strategy from Dev to QA and Ops
• Ways teams can collaborate to ensure top application performance

Publié dans : Technologie
  • Identifiez-vous pour voir les commentaires

Continuous Performance Testing and Monitoring in Agile Development

  1. 1. Continuous Performance Testing and Monitoring in Agile Development
  2. 2. Deliver faster applications faster through continuous performance validation Who We Are
  3. 3. Who We Are Mission: Deliver faster applications faster through continuous performance validation
  4. 4. Who We Are Mission: Deliver faster applications faster through continuous performance validation
  5. 5. User Demands Process Revolutions We’re Driven by Your Challenges Technology Transitions
  6. 6. AGENDA Performance matters Agile testing From Agile to DevOps DevOps pipeline Dynatrace Integration
  7. 7. 40% of users are moving towards the competition following a bad experience 57 % of users use their mobile 44-61 % of users share their feelings on social network User Experience is Key
  8. 8. Usability Ergonomic Performance Accessibility Features What is User Experience?
  9. 9. Etam reduces it's average page load time from 1.2s to 500ms and increased conversions by 20%, time on site by 21%, and pages viewed per visit by 28%. Walmart saw up to a 2% increase in conversions for every 1 second of improvement in load time. Every 100ms improvement also resulted in up to a 1% increase in revenue. Shopzilla decreased load time by 5 seconds and saw a 12% increase in conversion rate, a 25% increase in page views and a 50% reduction in infrastructure required. Etsy saw a 12% increase in bounce rate when they added 160kb of images to their mobile page. User Experience Impact on Business
  10. 10. AGENDA Performance matters Agile testing From Agile to DevOps DevOps pipeline Dynatrace Integration
  11. 11. Performance Testing Used To Be ….
  12. 12. Performance Design is The Key
  13. 13. Standard Performance Testing Workflow Strategy Scenarios Prepare Execute AnalyzeTuning
  14. 14. Testing Earlier Early Risk analysis • Understand the application • Performance needs to be included in the PRA • ATAM ( Architecture Tradeoff Analysis Method) Automation Smarter and faster Validation Follow our KPI Monitor Detect problems before the end users
  15. 15. Performance is driven by : • The end users • Third party system • … We need to ask the right questions : • How the users work on the system? • What are their habits • When ? and how often • Are we going to expand in different geos? • Is the marketing plan to market/promote the application? If yes what is the type audience? … Take The Time To Understand The Application
  16. 16. Performance testing an existing application • Ask the support of functional architect, HR..etc • Understand the behavior of the application with the help of the logs Migration project : Don’t underestimate the history of the application Performance testing a new application/service • Involve the project leader , functional architect • Try to understand the purpose and relation with the business plan Testing scenarios
  17. 17. Unit Performance Testing Any part of the system Not a standard practice Do not wait until the system is assembled Test cases are simpler, fewer variables • Test-Driven Development may be an answer Many systems are monolithic Third-party components
  18. 18. Change Mentality Before  Late record/playback performance testing  System-level requirements  Record/playback approach  "Black Box" Now  Early Performance Engineering  Component-level requirements  Programming to generate load/create stubs  "Grey Box”
  19. 19. Workflow 3) Development life cycle Component Testing Business Test Case 1) Specification 2) Risk Analysis Project Lifecycle
  20. 20. Neotys Continuous Performance Validation Solution Component testing Requirements Daily Cycle Performance End-to-End testing Sprint Assemble and Deployment Feedback Production Monitoring Performance Qualification Continuous Integration
  21. 21. AGENDA Performance matters Agile testing From Agile to DevOps DevOps pipeline Dynatrace Integration
  22. 22. Organizations Adopt Agile and DevOps as an Answer • 99% of organizations have adopted Agile development methods • 88% of CIOs are using DevOps World Quality Report 2016-17
  23. 23. But Speed is Nothing Without Quality “Surviving Disruption, Leading Change: Winning in the Application Economy,” 2015 2/3 of business leaders say their company’s future DEPENDS on the QUALITY of their software
  24. 24. “The term everyone is using is DevOps, but I think it should really be DevTestOps. Testing is really crucial for achieving quality & speed” Diego La Guidice, Principal Analyst DevTestOps Rather Than DevOps
  25. 25. AGENDA Performance matters Agile testing From Agile to DevOps DevOps pipeline Dynatrace integration
  26. 26. Planning • Provide transparency to the stakeholders Source Control • Tracking Change of the code ( infrastructure, application..Etc) Configuration management • Infrastructure is treated exactly like code Continuous integration • Tools to automate the Build and the deployment Deployment tools • Application can be released to production any time you want in order to improve time to market Testing and validation • Validate the quality of the release APM • applications are commonly tested and monitored with APM tools to ensure high availability, low response time, and quality of service DevOps Toolchain
  27. 27. Stage 1: Build Build request Prepare Build Compile Packaging
  28. 28. Stage 2: Deploy Deploy DEV QA UAT
  29. 29. Stage 3: Test receive task integration test DEV QA UAT Test Integration Functional Performance Security
  30. 30. Stage 4 : Release Release PROD
  31. 31. receive task integration test Devops toolchain Build request Deploy Release DEV QA UAT PROD Test Prepare Build Compile Packaging Integration Functional Performance Security
  32. 32. AGENDA Performance matters Agile testing From Agile to DevOps DevOps pipeline Dynatrace integration
  33. 33. • response time • infrastructure behavior Capture performance metrics that go beyond • Failures, bottlenecks, slower-than-expected response times Actionable data in case of problems Why Combine Load Testing And APM During a Load Test?
  34. 34. How Does the Integration Works? Web Model Mobile Model Rest Interface Business logic Data Access logic XML Legacy Relationnal Presentation Business Data NeoLoad Virtual Users Load
  35. 35. • Transaction name, virtual user, script name, unique id • Easier correlation between load testing solution and Dynatrace AppMon Web request tagging • Start & stop recording of data to cover exactly one test run Session Recording • Each web request is tagged with a test id • Allows to distinguish between requests if two test are run in parallel Registering individual test executions NeoLoad/Dynatrace Integration
  36. 36. • Download our free version of NeoLoad including the integration with Dynatrace Start LoadTesting with NeoLoad now • Contact us Ask for a personalized demo or a dedicated workshop Call for actions
  37. 37. Next Steps henrik.rexed@neotys.com Visit Neotys Community at answers.neotys.com Download NeoLoad Free Edition from www.neotys.com Questions? Visit neotys.com Follow us on Twitter @hrexed
  38. 38. mobile browser network multi-geo 3rd parties cloud containers services code hosts synthetic logs business transaction applications sdn relax full-stack, broad, hyperscale
  39. 39. Synthetic Agents Deployed and managed by Dynatrace Backbone Last Mile Web Performance Management 100+ locations Web Performance Management and Load Testing 10,000’s+ locations Mobile Dozens of countries connected to real wireless carriers Global Performance Network
  40. 40. Database and Mainframe 1st Tier N-Tier Front End 0.2s App Tier 1.1s Middleware 0.3s Database 0.1s Login Search Order Stock NETWORK ENTERPRISE APPS Agentless Monitoring
  41. 41. Inspect single Users Biz/App User Experience Management
  42. 42. Deep Transaction Analysis
  43. 43. It‘s not about blind automation of pushing more bad code through a shiny pipeline
  44. 44. It‘s not about blindly giving everyone Ops power to deploy changes only tested locally
  45. 45. Level-Up your Functional Tests with Metrics Build 17 testPurchase OK testSearch OK Build # Test Case Status Test & Monitoring Framework Results Test/Arch
  46. 46. Level-Up your Functional Tests with Metrics Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build # Test Case Status Test & Monitoring Framework Results We identified a regression Test/Arch
  47. 47. Level-Up your Functional Tests with Metrics Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status Test & Monitoring Framework Results Problem solved Test/Arch
  48. 48. Level-Up your Functional Tests with Metrics Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU Test & Monitoring Framework Results Architectural Data Let’s look behind the scenes Test/Arch
  49. 49. Level-Up your Functional Tests with Metrics Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU 12 0 120ms 3 1 68ms Test & Monitoring Framework Results Architectural Data Let’s look behind the scenes Test/Arch
  50. 50. Level-Up your Functional Tests with Metrics 12 0 120ms 3 1 68ms Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU 12 0 120ms 3 1 68ms 12 5 60ms 3 1 68ms 75 0 230ms 3 1 68ms Test & Monitoring Framework Results Architectural Data Exceptions probably reason for failed tests Let’s look behind the scenes Test/Arch
  51. 51. Level-Up your Functional Tests with Metrics 12 0 120ms 3 1 68ms Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU 12 0 120ms 3 1 68ms 12 5 60ms 3 1 68ms 75 0 230ms 3 1 68ms Test & Monitoring Framework Results Architectural Data Problem fixed but now we have an architectural regression Problem fixed but now we have an architectural regression Let’s look behind the scenes Test/Arch
  52. 52. Level-Up your Functional Tests with Metrics 12 0 120ms 3 1 68ms Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU 12 0 120ms 3 1 68ms 12 5 60ms 3 1 68ms 75 0 230ms 3 1 68ms Test & Monitoring Framework Results Architectural Data Now we have the functional and architectural confidence Let’s look behind the scenes Test/Arch
  53. 53. #1: Analyzing every Unit, Integration & REST API test #2: Key Architectural Metrics for each test #3: Detecting regression based on measure per Checkin Add Metrics into Continuous Integration Arch/CI
  54. 54. Quality Overview by Build In Dynatrace … #8: Integrate into your Delivery Pipeline CI/CD
  55. 55. Quality Overview by Build In Dynatrace … … allows you to Stop a Bad Build in Jenkins, NeoLoad, … #8: Integrate into your Delivery Pipeline CI/CD
  56. 56. One goal: deliver better features to customers faster Two fundamental components: speed + quality
  57. 57. Take the next step with a Dynatrace Free Trial !!! dynatrace.com/trial Sign up at:
  58. 58. Confidential, Dynatrace, LLC Q & A Henrik Rexed Performance Engineer @hrexed Asad Ali Product Specialist @AsadThoughts Download NeoLoad Free Edition at: www.neotys.com Try Dynatrace for FREE: www.dynatrace.com/trial

×