Moving Towards Zero Defects with Specification by Example
•Télécharger en tant que PPTX, PDF•
4 j'aime•6,932 vues
Love tracing bugs in a defect tracking system? Love the bug-fix cycle? If so, then don't come to this presentation. We'll be discussing how Specification by Example (also known as Acceptance Test Driven Development) will help move you towards a zero defect system by building the right thing the first time.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Moving Towards Zero Defects with Specification by Example
Similaire à Moving Towards Zero Defects with Specification by Example (20)
Plus de Steve Rogalsky
Plus de Steve Rogalsky (18)
Dernier
Dernier (20)
Moving Towards Zero Defects with Specification by Example
- 1. (Track Sponsor) Moving towards zero defects with Specification By Example Steve Rogalsky @srogalsky winnipegagilist.blogspot.com
- 2. Choose 2 of these topics to discuss: Talk about your approach to quality and the results of that approach. Talk about your approach to requirements and how you keep those requirements up to date. Talk about any previous experience with automated testing. Talk about your team’s current bug tracking process. Talk about zero defects – Is it possible? Is it responsible? Cost effective?
- 5. A story of Zero Defects
- 7. HYPOTHESIS: Specification By Example improves Quality & Speed
- 8. DEFINITIONS: Acceptance Testing Is...
- 9. What is Specification By Example? Goal: To build the right thing the first time.
- 10. What is Specification By Example? Also known as / similar to: ATDD – Acceptance Test Driven Development BDD – Behaviour Driven Development
- 11. To do this we: 1. WRITE EXAMPLES (Acceptance Tests) (up front but not UP FRONT) instead of requirements Given muppet <Animal> When measuring <Craziness> Then return <10> Given muppet <Animal> When <Drumming> Then return <Phenomenal Skillz> Given muppet <Animal> When <talking> Then return <Grunt> SPECIFICATION BY EXAMPLE
- 12. To do this we: 2. TEST AS SOON AS POSSIBLE FIRST in collaboration with the developers and customers
- 14. To do this we: 3. AUTOMATE our examples
- 15. AN EXAMPLE: Requirement: Bring home something small from Europe What I brought: What she wanted:
- 16. ANOTHER EXAMPLE User Story: As an employee I want to receive overtime pay For each week, hourly employees are paid: 2 times their wage for each hour worked on Sundays and holidays a standard wage per hour for the first 40 hours worked 1.5 times their wage for each hour after the first 40 hours
- 17. (40*$20) = $800.00 a standard wage per hour for the first 40 hours worked
- 18. (40*$20) + (5*$20*1.5) = $950.00 1.5 times their wage for each hour after the first 40 hours
- 19. 2 times their wage for each hour worked on Sundays and holidays (40*$20) + (8*$20*1.5) + (8*$20*2) = $1,360.00
- 20. 2 times their wage for each hour worked on Sundays and holidays (40*$20) + (8*$20*1.5) + (8*$20*2 ) = $1,360.00 *1.5
- 21. Executable Specifications Credit: This example is from “The Role of Quality Assurance in Lean-Agile” – Allan Shalloway
- 22. Defect Tracking
- 25. Where should we focus our automated testing? Later… Focus Here First UI Services TDD Classes/ Functions Credit: Testing Triangle first shown by Patrick Wilson-Welsh
- 26. What does it take to set this up? It is simpler than this… '''
- 27. Steps: Download FitNesse (free) Run FitNesse Add A Reference to your project Create a Fixture per set of examples Write the examples Press a button
- 28. How to do it? Tester Customer and Team Automate Examples Expand into Examples Choose Story High Level Tests Passes All tests = done Review Code / TDD Think about how Developer
- 29. THE EXPERIMENT
- 34. TO SUMMARIZE Specification By Example: Communicate! Write Executable Examples instead of requirements The Tooling is simple to use and free Zero Defects isn’t impossible – build it right the first time Trash your Defect Tracker
- 35. THANKS! Questions? Links Contact Info http://gojko.net/fitnesse/ steve.rogalsky@protegra.com http://fitnesse.org/ @srogalsky winnipegagilist.blogspot.com http://seleniumhq.org/ http://www.slideshare.net/SteveRogalsky/moving-towards-zero-defects-with-specification-by-example
Notes de l'éditeur
- SETUPFitNesse example- Test that these work and are prepped and then close them down: - Run C:\\_Projects\\SampleProjects\\SpecByExample\\server.FitNesse.bat - Open http://localhost:8088 in FireFox - Confirm tests are still all passing. - In FitNesse, remove page SpeakerInformation- Open C# project C:\\_Projects\\SampleProjects\\SpecByExample\\PrairieDevCon\\PrairieDevCon.sln - In VS, open "CalculateWeeklyPay" - In VS, remove LookupSpeakerInformation.cs- Setup 2 tables at the front for the experiment- Open C:\\_Projects\\FitNesse\\ folderOpen script: C:\\Users\\srogalsky\\Dropbox\\Presentations\\PrDC11\\SpecificationByExample\\ScriptForDemo.docx2. ExperimentPut out paperSetup 2 tablesSetup dividersPut examples in the dividersOpen spreadsheet: C:\\Users\\srogalsky\\Dropbox\\Presentations\\PrDC11\\SpecificationByExample\\AirplaneMfgResults.xlsPut instructions on both side – for QA, for team
- Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
- Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
- Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
- Introduce myself here first.Tell my story here of the first project using this. 6 defects in 12 person months of work. 3 mediums, 3 lows. Arguably no defects since the 6 found were not identified as tests.Points:Cost?Is there a cost to TDD?What is the cost of a defect? (have you seen that chart?)The later you find it, the more screwed you are.Defects that matter“Like” button vs FB securityBuild it right the first timeBuild each piece little by little. Perfect on top of perfect. Simple…Normalization of devianceBoiling the frogIf you are using iterations, you need to regression test each iterationHow can you afford not to do some automated examples? Do you want to regression test your whole app manually at the end of each iteration? After each story is completed?Click a buttonZero defects is going be to required on more and more projects.Google carSmart cars that self regulate speed in car pool lanes once you enter themLots of Health initiativesWe’re moving to be ‘more’ dependent on perfect apps, not lessThe difference between what you think you should do and what you actually do is the degree to which you suckCory Haines
- HypothesisDefinitionsProof
- The stuff that QA and the customer does before saying it is done.Functional TestingAcceptance TestingNot Unit TestingWe’re not talking about TDD (Test Driven Development) today. Although related, TDD applies more to design, flexible code and unit testing. We’re talking about testing at a higher level – customer, QA, etc.
- Happy customersRemove frustrations
- Happy customersRemove frustrations
- Makes requirements less ambiguous – an example laterGWT example shown, but lots of ways to do this – start with your existing test case formats and see what works for you.If you can’t do the rest of ATDD, do this! A great place to start.
- Catches misunderstandings early so they aren’t duplicated throughout other user storiesImproves communication between QA and developers and customers (whole team!)Reduces the time spent writing, reading, understanding, arguing items in a defect trackerWhen we test, we execute the examples together that we wrote earlier
- This is a silly as a high school end of year exam that is worth 60%. If we fail, it is as much the teacher’s fault as ours.
- To prove early that the system works as expected and eliminate the waste of reworkRequires and investment time up front required to write the tests – pay off is laterRegression testing effort disappears (click a button)
- To prove early that the system works as expected and eliminate the waste of reworkRequires an investment of time up front to write the tests – pay off is laterRegression testing effort disappears (click a button)
- example courtesy Allan Shalloway - The Role of Quality Assurance in Lean-Agile
- An example using FitNesseWe run this test against the code and see it passes!
- Now with some overtime
- Now with overtime and holiday hours
- Now with the corrected codeThis is the beginning of the “aha” moment
- Putting it all together – this is what your executable requirements document now looks like.Complete traceabilityPush button testing and regression now existsCan replace all business logic in your requirements documentCan replace much of your bug tracking effortThis should be the ‘aha’ moment.
- Show Quick example using C#, FitNesse, PrDC APIStart the fitnesse batch fileShow the added referenceShow the fixture code for “CalculateWeeklyPay”Open FitNesse (http://localhost:8088/)Show the markup language for “WeeklyPayWithOvertime”Add an example for 40 hours and 10/hour = 500Press the buttonFix the errorShow HistoryShow the suiteRun the suiteShow the defect we found and fixed in the example earlier.Comment: This was my own biggest hurdle – starting…
- Janet Gregory’s slide modified. Talk about the process. Talk about not Up Front, but just in time – one story at a time.Follow this little roll play:Script:=========Choose story (create a new FitNesse page and put this in there): Create “SpeakerInformation” pageAs an attendee I want to know some information about a speaker so that I can make decisions on which sessions to attend2. High Level Tests (add to FitNesse page):a short bio* name and contact info* social media information3. Expand into examples:'''User Story: Speaker Information'''''As an attendee I want to know some information about a speaker so that I can make decisions on which sessions to attend''* a short bio* name and contact info* social media information'''Scenario: Show the contact information for a speaker'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!Given a '''<Speaker Name>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!When I look up their information on the website!-&nbsp;&nbsp;&nbsp;&nbsp;-!Then it should return the '''<Email>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!And '''<Twitter Id>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!And '''<Blog>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!And '''<Company Name>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!And '''<Picture>''''''Examples:'''!|Lookup Speaker Information ||Speaker Name |Email? |Twitter Id?|Blog? |Company Name?|Picture? ||Steve Rogalsky|steve.rogalsky@protegra.com|@srogalsky |winnipegagilist.blogspot.com/|Protegra |steverogalsky.jpg|4. Review together5. Automate examplesusing System;using System.Collections.Generic;using System.Linq;using System.Text;namespace PrairieDevCon.FitnesseTests{ public class LookupSpeakerInformation : fit.ColumnFixture { public string SpeakerName; public string TwitterId; public string Blog; public string CompanyName; public string Picture; public string Email() {varspeakerInformation = PrairieDevCon.Services.SpeakerService.GetSpeakerByName(SpeakerName);TwitterId = speakerInformation.Twitter; Blog = speakerInformation.Blog;CompanyName = speakerInformation.Company; Picture = speakerInformation.Picture; return speakerInformation.Email; } }}6. Run the tests until done!
- At this point we gather our volunteers and explain the experiment.Explain:First, the teams may not talk from this point forwardQAPreferably acted out by a non-QA person(ask who are the testers, and pick a non-tester)Will accept/reject the end productHas a little leeway in accepting – see the ‘test cases’ provided for you. As long as it is pretty close, you can accept it. I am your customer, so I can help you with that.If it isn’t right, please write up the defect on your paper using words or pictures and hand the defect log and the product back to the develop team to fixAsk the QA to person to write good defects – as a dev he/she will know how frustrating an unclear defect isManufacturing team = DevelopersThere will be a few steps to build the end productThese steps are similar to the steps that you create to build software – the classes, database tables, UI, layers, etc. QA won’t test the individual classes and steps, but they will perform functional or acceptance testing on the final productYou can pair program if you like, it is up to you to determine the best way to accomplish the task – but again, NO TALKINGTeam 1Will not use ATDD – they will test at the end and to simulate this they will not see the ‘test cases’ that QA ownsTeam 2Will use ATDD – they will create their test case at the beginning to simulate the effort of automating the tests up front and they will have access to the ‘test cases’ throughout developmentBoth teamsReminder – no talkingReminder – no fighting or blaming – everyone is doing the best they can with what they have been givenDon’t copy the other team’s product, the requirements are similar, but the test cases may not be the sameTry to be as successful as possible without cheatingIf you have a defect, don’t throw out the product and start fresh, but fix the defective productWe’ll be timing both teams and then examine the results through a value stream before we debrief with questions
- Once the exercise is done:First: Fill out the spreadsheetSecond: Ask the observers to reportThird – ask questions:How did it go?How did you feel as you were developing (calm on one side, not on the other)What went well?What didn’t go well?Refer back to the original questions asked and their answers – how does this fit?Fourth: my observationsHow it related to agile: removing barriers; build the right thing the first time; fail fast not slow