Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
www.planittesting.com
BDDP
A Discussion on Behaviour Driven
Development for Performance
Joel Deutscher
Performance Practic...
www.planittesting.com © Planit Software Testing 2
Continuous Performance
Testing
www.planittesting.com © Planit Software Testing 3
Meaningful Continuous
Performance Testing
And can BDD get us there?
www.planittesting.com
My Background
© Planit Software Testing 4
www.planittesting.com © Planit Software Testing 5
It’s my opinion
Which is very likely to be incorrect
www.planittesting.com
Why can’t I have a big go button?
© Planit Software Testing 6
www.planittesting.com
There are lots of excuses!
© Planit Software Testing 7
www.planittesting.com
A typical release based process
© Planit Software Testing 8
www.planittesting.com © Planit Software Testing 9
So what could we do?
We need to start somewhere
www.planittesting.com
Automated Execution and Reporting
© Planit Software Testing 10
www.planittesting.com
We actually created a go button!
© Planit Software Testing 11
www.planittesting.com
Automated Reporting – SLA Based
© Planit Software Testing 12
www.planittesting.com
Automated Reporting - Baseline
© Planit Software Testing 13
www.planittesting.com
Automated Pass / Fail Notifications
© Planit Software Testing 14
www.planittesting.com
Test Preparation
© Planit Software Testing 15
www.planittesting.com © Planit Software Testing 16
How do we get to continuous
performance testing?
… and make it meaningf...
www.planittesting.com
So what is left?
© Planit Software Testing 17
www.planittesting.com © Planit Software Testing 18
What are the cool kids doing?
www.planittesting.com
TDD
Did we build the system right?
BDD
Did we build the right system?
© Planit Software Testing 19
www.planittesting.com
“TDD took extra time upfront the resulting quality
was higher than teams that adopted a non-TDD
appr...
www.planittesting.com
Extra Time Upfront?
© Planit Software Testing 21
www.planittesting.com © Planit Software Testing 22
Why isn’t performance
considered earlier?
Requirements
www.planittesting.com
• Functional Requirements are usually terrible
• Non-Functional Requirements are worse!
• Garbage in...
www.planittesting.com
• Requirements gathering is the hardest part of most
performance tests
• Manual and laborious proces...
www.planittesting.com © Planit Software Testing 25
Why are they so bad?
www.planittesting.com
• Requirements are written by people with no idea of what they want
• Performance is not well unders...
www.planittesting.com © Planit Software Testing 27
OK Google, What are some
good NFRs?
This should be good…
www.planittesting.com
“System Y shall process a minimum of 8 transactions per
“second”
and
“The system must handle 1,000 t...
www.planittesting.com © Planit Software Testing 29
What's wrong with that?
Where do I begin.
www.planittesting.com
The system must handle 1,000 transactions per second
• What should the response times should there b...
www.planittesting.com
“System Y shall process a minimum of 8 transactions per second”
and
“The system must handle 1,000 tr...
www.planittesting.com © Planit Software Testing 32
How do you automate
performance testing without
good requirements?
www.planittesting.com
Either through Non-Functional Requirements or Server Logs we
tend to get the information like:
We wa...
www.planittesting.com © Planit Software Testing 34
Oh, and everything should be
as fast as the current release
… which is?
www.planittesting.com © Planit Software Testing 35
Could clear requirements
defined early be the answer?
www.planittesting.com
The internet banking platform must:
• support 1000 valid user logins per hour
• with a 95 percentile...
www.planittesting.com © Planit Software Testing 37
User Stories Can help
www.planittesting.com
As a <type of user>,
I want <some goal>
so that <some reason>
© Planit Software Testing 38
User Stor...
www.planittesting.com
As an Account Holder,
I want to login to internet banking
So that I can do banking on the internet
©...
www.planittesting.com © Planit Software Testing 40
User Stories for NFRs
www.planittesting.com
As an Account Holder,
I want to login to internet banking within 3 seconds
So that I can transact on...
www.planittesting.com
Title (one line describing the story)
Narrative:
As a [role]
I want [feature]
So that [benefit]
Acce...
www.planittesting.com © Planit Software Testing 43
Its in the acceptance criteria?
But how detailed should we get for NFRs?
www.planittesting.com
Scenario: Submit the contact form with correct data (declarative)
Given I am on the company contact ...
www.planittesting.com
Scenario: Submit the contact form with correct data (imperative)
Given I am on the company contact p...
www.planittesting.com © Planit Software Testing 46
Imperative
www.planittesting.com
Scenario: Submit the contact form with correct data (imperative)
Given I am on the company contact p...
www.planittesting.com © Planit Software Testing 48
Problem Solved?
Now we just need to automate it….
www.planittesting.com
PROS
• Performance Considered Early
• Developers can Design for
Performance
• Seems easier to specif...
www.planittesting.com © Planit Software Testing 50
Moon Shot?
www.planittesting.com
Scenario: Weather Service
Given the webservice named Weather
When I call the method GetCurrentTemper...
www.planittesting.com © Planit Software Testing 52
Meaningful Continuous
Performance Testing
Can BDD help get us there?
www.planittesting.com
• Repeatable Environment
• Managing Test Data
• SLA vs Baseline Comparison
• Trend Analysis
© Planit...
www.planittesting.com
• BDD for Performance is still a work in progress
• User Story / Acceptance Test Format is not ideal...
www.planittesting.com © Planit Software Testing 55
What has been your
experience?
Please come and speak to me throughout t...
Prochain SlideShare
Chargement dans…5
×

BDDP: Behaviour Driven Development for Performance (NZ Tech Day - Presentation 1)

Non-functional requirements are often misunderstood, overly technical or missing altogether. This results in systems that are not designed for performance, and performance tests with no real alignment to the business. Behaviour Driven Development has helped teams to document requirements in fast-paced Agile environments. Clearly defined requirements early in the project are the starting point that allows test automation to be included in a Continuous Integration (CI) environment.

Behaviour-Driven Development for Performance encourages a simplified approach to documenting requirements from an end user perspective with common sense terminology and is the first step on the journey to integrating performance testing into a CI workflow.

For more information about our Performance Testing services, visit our website:
https://www.planittesting.com/Services/Performance-Testing

  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

BDDP: Behaviour Driven Development for Performance (NZ Tech Day - Presentation 1)

  1. 1. www.planittesting.com BDDP A Discussion on Behaviour Driven Development for Performance Joel Deutscher Performance Practice Lead jd@planittesting.com © Planit Software Testing 1
  2. 2. www.planittesting.com © Planit Software Testing 2 Continuous Performance Testing
  3. 3. www.planittesting.com © Planit Software Testing 3 Meaningful Continuous Performance Testing And can BDD get us there?
  4. 4. www.planittesting.com My Background © Planit Software Testing 4
  5. 5. www.planittesting.com © Planit Software Testing 5 It’s my opinion Which is very likely to be incorrect
  6. 6. www.planittesting.com Why can’t I have a big go button? © Planit Software Testing 6
  7. 7. www.planittesting.com There are lots of excuses! © Planit Software Testing 7
  8. 8. www.planittesting.com A typical release based process © Planit Software Testing 8
  9. 9. www.planittesting.com © Planit Software Testing 9 So what could we do? We need to start somewhere
  10. 10. www.planittesting.com Automated Execution and Reporting © Planit Software Testing 10
  11. 11. www.planittesting.com We actually created a go button! © Planit Software Testing 11
  12. 12. www.planittesting.com Automated Reporting – SLA Based © Planit Software Testing 12
  13. 13. www.planittesting.com Automated Reporting - Baseline © Planit Software Testing 13
  14. 14. www.planittesting.com Automated Pass / Fail Notifications © Planit Software Testing 14
  15. 15. www.planittesting.com Test Preparation © Planit Software Testing 15
  16. 16. www.planittesting.com © Planit Software Testing 16 How do we get to continuous performance testing? … and make it meaningful
  17. 17. www.planittesting.com So what is left? © Planit Software Testing 17
  18. 18. www.planittesting.com © Planit Software Testing 18 What are the cool kids doing?
  19. 19. www.planittesting.com TDD Did we build the system right? BDD Did we build the right system? © Planit Software Testing 19
  20. 20. www.planittesting.com “TDD took extra time upfront the resulting quality was higher than teams that adopted a non-TDD approach by an order of at least two times.” Evaluating the Efficacy of Test-Driven Development: Industrial Case Studies Thirumalesh Bhat and Nachiappan Nagappan © Planit Software Testing 20
  21. 21. www.planittesting.com Extra Time Upfront? © Planit Software Testing 21
  22. 22. www.planittesting.com © Planit Software Testing 22 Why isn’t performance considered earlier? Requirements
  23. 23. www.planittesting.com • Functional Requirements are usually terrible • Non-Functional Requirements are worse! • Garbage in, Garbage Out © Planit Software Testing 23 Requirements
  24. 24. www.planittesting.com • Requirements gathering is the hardest part of most performance tests • Manual and laborious process • Why do we need to manually evaluate the risk if its properly defined in requirements? © Planit Software Testing 24 Why does it matter?
  25. 25. www.planittesting.com © Planit Software Testing 25 Why are they so bad?
  26. 26. www.planittesting.com • Requirements are written by people with no idea of what they want • Performance is not well understood and is either: • Assumed or considered the "Of course" stuff • Considered but not well defined “I want it to be fast” • Considered but of no actual benefit “Must not be slower than the existing platform” • Considered but pointless “I want it as fast as the industry average” • Not considered at all © Planit Software Testing 26 Its Complicated
  27. 27. www.planittesting.com © Planit Software Testing 27 OK Google, What are some good NFRs? This should be good…
  28. 28. www.planittesting.com “System Y shall process a minimum of 8 transactions per “second” and “The system must handle 1,000 transactions per second” © Planit Software Testing 28 This is what I found…
  29. 29. www.planittesting.com © Planit Software Testing 29 What's wrong with that? Where do I begin.
  30. 30. www.planittesting.com The system must handle 1,000 transactions per second • What should the response times should there be when it hits 1,000 transactions per second • What type of transactions? • Is it OK if the CPU maxes out to 100% while this is happening? • How long should it be able to handle 1,000 per second for? • Where should this be measured? • What is an acceptable error rate if any? © Planit Software Testing 30 Its not serving its purpose
  31. 31. www.planittesting.com “System Y shall process a minimum of 8 transactions per second” and “The system must handle 1,000 transactions per second” © Planit Software Testing 31 Where did I find these examples? “System Y shall process a minimum of 8 transactions per second” and “The system must handle 1,000 transactions per second”
  32. 32. www.planittesting.com © Planit Software Testing 32 How do you automate performance testing without good requirements?
  33. 33. www.planittesting.com Either through Non-Functional Requirements or Server Logs we tend to get the information like: We want to run a test that has: • 1000 logins per hour • 800 Transaction History Views per Hour • 300 Fund Transfers • 50 BPay Transactions • 5 Credit Card Applications © Planit Software Testing 33 Volumetrics are not requirements
  34. 34. www.planittesting.com © Planit Software Testing 34 Oh, and everything should be as fast as the current release … which is?
  35. 35. www.planittesting.com © Planit Software Testing 35 Could clear requirements defined early be the answer?
  36. 36. www.planittesting.com The internet banking platform must: • support 1000 valid user logins per hour • with a 95 percentile response time of 1 second • with a maximum response time of 3 seconds • with response times measured at the web server • with a maximum error rate of 1% • with peak platform usage occurring on the system • with system resource utilisation not breaching 60% on any component © Planit Software Testing 36 Login Page NFR
  37. 37. www.planittesting.com © Planit Software Testing 37 User Stories Can help
  38. 38. www.planittesting.com As a <type of user>, I want <some goal> so that <some reason> © Planit Software Testing 38 User Stories
  39. 39. www.planittesting.com As an Account Holder, I want to login to internet banking So that I can do banking on the internet © Planit Software Testing 39 User Stories
  40. 40. www.planittesting.com © Planit Software Testing 40 User Stories for NFRs
  41. 41. www.planittesting.com As an Account Holder, I want to login to internet banking within 3 seconds So that I can transact on the internet © Planit Software Testing 41 User Stories
  42. 42. www.planittesting.com Title (one line describing the story) Narrative: As a [role] I want [feature] So that [benefit] Acceptance Criteria: (presented as Scenarios) Scenario 1: Title Given [context] And [some more context]... When [event] Then [outcome] And [another outcome]... © Planit Software Testing 42 User Stories and Acceptance Criteria
  43. 43. www.planittesting.com © Planit Software Testing 43 Its in the acceptance criteria? But how detailed should we get for NFRs?
  44. 44. www.planittesting.com Scenario: Submit the contact form with correct data (declarative) Given I am on the company contact page When I submit the form with correct data Then I should see the correct message © Planit Software Testing 44 A declarative (informative) example
  45. 45. www.planittesting.com Scenario: Submit the contact form with correct data (imperative) Given I am on the company contact page When I set "What is your name?" to “Joel" And I set “What is your query to?" to “I wish to discuss performance testing" And I set "What is your email address to “jd@planittesting.com" And I click "Submit" Then I should see the message "Thanks! The Planit team will be in contact shortly." © Planit Software Testing 45 An imperative (communicative) example
  46. 46. www.planittesting.com © Planit Software Testing 46 Imperative
  47. 47. www.planittesting.com Scenario: Submit the contact form with correct data (imperative) Given I am on the company contact page with 1000 concurrent users on the site When I set "What is your name?" to “Joel" And I set “What is your query to?" to “I wish to discuss performance testing" And I set "What is your email address to “jd@planittesting.com" And I click "Submit" Then I should see the message "Thanks! The Planit team will be in contact shortly.“ And the response should be returned within 2 seconds 95 percent of the time And the response should not take more than 5 seconds to return And the error rate should be less than 1% And the server utilisation should be less than 60% for all components © Planit Software Testing 47 Scenario with Performance Requirements
  48. 48. www.planittesting.com © Planit Software Testing 48 Problem Solved? Now we just need to automate it….
  49. 49. www.planittesting.com PROS • Performance Considered Early • Developers can Design for Performance • Seems easier to specify CONS • Its not exactly elegant • Potentially gets in the way of delivering functionality • Very easy to get wrong © Planit Software Testing 49 Its not exactly simple
  50. 50. www.planittesting.com © Planit Software Testing 50 Moon Shot?
  51. 51. www.planittesting.com Scenario: Weather Service Given the webservice named Weather When I call the method GetCurrentTemperature with a valid postcode 1000 times in an hour Then I expect it to return an containing the current temperature And the response should be returned within 2 seconds 95 percent of the time And the response should not take more than 5 seconds to return And the error rate should be less than 1% And the server utilisation should be less than 60% for all components © Planit Software Testing 51 Weather Micro Service
  52. 52. www.planittesting.com © Planit Software Testing 52 Meaningful Continuous Performance Testing Can BDD help get us there?
  53. 53. www.planittesting.com • Repeatable Environment • Managing Test Data • SLA vs Baseline Comparison • Trend Analysis © Planit Software Testing 53 There are some other considerations
  54. 54. www.planittesting.com • BDD for Performance is still a work in progress • User Story / Acceptance Test Format is not ideal in every situation • Works well for simple web services, less so for socialised performance tests • System Resource Utilisation Metrics are important, though shouldn’t be in every acceptance scenario © Planit Software Testing 54 The Story has Just Begun
  55. 55. www.planittesting.com © Planit Software Testing 55 What has been your experience? Please come and speak to me throughout the day

    Soyez le premier à commenter

    Identifiez-vous pour voir les commentaires

Non-functional requirements are often misunderstood, overly technical or missing altogether. This results in systems that are not designed for performance, and performance tests with no real alignment to the business. Behaviour Driven Development has helped teams to document requirements in fast-paced Agile environments. Clearly defined requirements early in the project are the starting point that allows test automation to be included in a Continuous Integration (CI) environment. Behaviour-Driven Development for Performance encourages a simplified approach to documenting requirements from an end user perspective with common sense terminology and is the first step on the journey to integrating performance testing into a CI workflow. For more information about our Performance Testing services, visit our website: https://www.planittesting.com/Services/Performance-Testing

Vues

Nombre de vues

730

Sur Slideshare

0

À partir des intégrations

0

Nombre d'intégrations

445

Actions

Téléchargements

0

Partages

0

Commentaires

0

Mentions J'aime

0

×