Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à So What Do Cucumbers Have To Do With Testing
Similaire à So What Do Cucumbers Have To Do With Testing (20)
Dernier
Dernier (20)
So What Do Cucumbers Have To Do With Testing
- 1. CoastNerds June 2011 So What Do Cucumbers Have To Do With Testing? Presented By Robert Dyball & Shannon Marsh
- 2. Behaviour Driven Development (BDD) BDD Tools StoryQ Demonstration How we use BDD & StoryQ in our Process CoastNerds Retrospective What are we going to talk about?
- 5. BDD = Behaviour Driven Development BDD
- 6. so a test case that looks like this:public class CustomerLookupTest extends TestCase{ testFindsCustomerById() { ... } testFailsForDuplicateCustomers() { ... } ... } rendered to something like this: CustomerLookup - finds customer by id - fails for duplicate customers - ...
- 7. Eric Evans: DDD – Domain Driven Design.
- 8. A User Story says: As a [X] I want [Y] so that [Z]
- 9. A User Story says: As a [X] I want [Y] so that [Z] BDD says: Given some initial context (the givens), Whenan event occurs, thenensure some outcomes.
- 10. Given, When, Then Ubiquitous Language:
- 11. TDD versus BDD So what do you do with TDD now?
- 12. BDD Tools
- 13. Cucumber is AslakHellesøy’s rewrite of RSpec’s “Story runner”, which was originally written by Dan North. Is a Ruby tool but can be used with Java, .Net, Flex (Adobe) or web based applications in written in any language. Tests are written in plain text using the Gherkin syntax. Cucumber only requires minimal use of Ruby programming. Cucumber Ref: http://cukes.info/ Ref: https://github.com/cucumber/cucumber/wiki/
- 14. Gherkin Syntax Ref: https://github.com/cucumber/cucumber/wiki/Gherkin
- 15. Cucumber Example More Examples: http://blog.adyax.com/2009/03/ruby-cucumber/ http://blog.spritecloud.com/2010/03/web-testing-with-cucumber/ https://github.com/cucumber/cucumber/tree/master/examples/i18n
- 16. Java based tool for BDD created by Dan North, et al. Uses plain text “Gherkin” syntax Java programming required Maps to Java based test using annotations Can integrate into a number of Java testing frameworks including JUnit. JBehave Ref: http://jbehave.org/reference/stable/index.html
- 17. JBehave Example
- 18. JBehave Example
- 19. JBehave Example Ref: http://jbehave.org/reference/stable/index.html
- 20. BDD testing framework for .Net Configurable to work with NUnit, MSTest, XUnit. Scenarios (specs) are defined in Gherkin syntax in a separate file. SpecFlow takes the scenarios and turns them into code stubs for tests to be created. (via command line utility) Scenarios can be data driven using a <placeholder> syntax. Provides HTML report with summary of all tests as well as details of each scenario. SpecFlow Ref: http://www.specflow.org/home.aspx
- 21. SpecFlow Example Ref: http://www.codeproject.com/KB/architecture/BddWithSpecFlow.aspx
- 22. SpecFlow Example Ref: http://www.specflow.org/home.aspx
- 23. BDD for JavaScript “JSpecis a extremely small, yet very powerful testing framework” Uses its own custom grammar and pre-processor (ie. Not Gherkin). Eg. Before_each, describe, it, should_have, end. Check-out the screen cast for a great demo of how it works and how to setup your tests http://content.screencast.com/users/tjholowaychuk/folders/Jing/media/816a8baa-0175-491a-a138-c889fff99f2d/00000077.swf JSpec Ref: http://visionmedia.github.com/jspec/
- 24. JSpec Example Ref: http://visionmedia.github.com/jspec/
- 25. JSpec Example Ref: http://visionmedia.github.com/jspec/
- 26. BDD for System Administrators Puppet is an enterprise system management platform. Allows you automate system management activities through code. Cucumber-Puppet has been developed to enable BDD for Puppet. Similarly Chef is an open-source system integration framework for automating the cloud. Cucumber-Puppet is a library of tools to enable TDD with Chef and lends it self to BDD. http://projects.puppetlabs.com/projects/cucumber-puppet Ref: http://blog.nistu.de/cucumber_puppet_0_0_3_released.html http://www.cucumber-chef.org/ https://github.com/jtimberman/chef-bdd
- 27. “Cucumber-chef begins with a very simple premise. If we are framing our infrastructure as code - if we’re writing cookbooks, recipes and other pieces of automation in a high level programming language, such as Ruby, then it makes sense to follow the current wisdom across the software development world to maximise the quality, maintainability and reusability of our code, providing maximum chance that we’ll deliver value with it….” Why use BDD for System Administration? http://www.cucumber-chef.org/ Ref:
- 28. Cucumber-Puppet Example Setup the fact/s Create the Test
- 29. StoryQ is a portable (single dll), embedded BDD framework for .NET 3.5 It runs within your existing test runner and helps produce human-friendly test output (html or text). StoryQ's fluent interface adds strong typing, intellisense and documentation to your BDD grammar. Report in plain text, test results or formated XML C# coding required. Featured in tonight’s Demo StoryQ Ref: http://storyq.codeplex.com/ http://storyq.codeplex.com/wikipage?title=write%20your%20first%20StoryQ%20test http://www.codeproject.com/KB/testing/bddintro.aspx
- 30. StoryQ Example
- 31. StoryQ Example
- 32. RSpec – Ruby - http://rspec.info/ CSpec – C - https://github.com/arnaudbrejeon/cspec/wiki easyb – Java - http://easyb.org/ storEvil – C# - https://github.com/davidmfoley/storevil/wiki/Why-StorEvil behat – PHP - http://behat.org/ Kiwi – Apple Objective C - http://www.kiwi-lib.info/ Lettuce – Python - http://packages.python.org/lettuce/tutorial/simple.html Others… Ref:
- 33. Demonstration BDD using StoryQ
- 34. Discussion: how we use BDD and StoryQ in our process BDD in use
Notes de l'éditeur
- BDD stands for Behaviour Driven DevelopmentBefore we start, a little about who we are, and what has motivated us to be here instead of watching a rerun of Seinfeld at home.Shannon and I have been working in a development team together for around 3 years, during this time we’ve used Agile / Scrum based methodology, built web applications, WPF / windows applications, and more recently worked in ETL development to migrate millions of data records from one organisation to another. While we don’t think we’re the best developers on earth, we’d like to share a few of the things we’ve learned in the hope we can improve the reputation of the profession of software development. We’ve all heard of the high failure rates in our industry of software development projects; projects run late, go over budget and then under deliver. This makes everyone nervous, we get nervous as developers – as we’ve seen poorly defined requirements. It makes project managers nervous as they see the user and developer causes of scope creep and testers, people who do not have an easy job either – balancing testing and pressures to deliver. None of this makes it an easy job for anyone.BDD can change this, we have been using this for 6 months now and can already see the benefits; we’ve been enjoying work more, find it less frustrating and more satisfying. The project owners are happier – they have a reliable product being produced and they have documentation.Now if you aren’t a programmer or developer, don’t get too worried, what we have for you on BDD will occasionally mention code, and we’ll have a short demonstration of the techniques, but we’ll also be showing you how BBD can now benefit people all through the software development life cycle as well as people in roles including server maintenance or support.Similarly if you are using waterfall or agile development methodologies like scrum, BDD can still work for you. And likewise, if you are a solo developer or part of a team, you will also find BDD can work.
- This is what we hope to avoid, and BDD has proven to us, to be a tool that can keep us from this.
- We’re first going to look at some history, how BDD came about, as we think you’ll see how it made sense then, and still makes sense. This will lead on into how BDD can be used.
- BDD is not “behaviour modification” - which is probably against the law, we’ve already said BDD stands for Behaviour Driven Development. But what is this?The term BDD was first used by Dan North in 2003. In the company where Dan worked, it was the usual practice to develop code using Test Driven Development. After seeing a few occasions in which TDD failed, Dan spotted a common thread.Dan had been using a utility called Agiledox, and while writing tests for his code was able to “parse out”, of filter, useful information from the test class names.
- Don’t get too fussed about the detail here, but look at the code on top, remove the word “Test” from the Class names, and you can get something that read like plain English.Dan used a convention of starting the class names of his tests with the word “should”. After thinking a little on why he was writing the software in the first place, he came up with the idea of “Whats’ the next most important thing the system doesn’t do?”. See: http://dannorth.net/introducing-bdd/
- Around 2004 Dan spotted similarities in where he had been heading when he saw Eric Evans work in DDD, or Domain Driven Design. Eric Evans spoke of DSL or Domain Specific Languages, and Ubiquitous Languages as ways to bridge the communication differences between developers, analysts, testers and product owners/stakeholders. Too often Eric saw project failures when one part of the team in the process did not speak the same language as another part.Dan North and colleague Chris Stevenson realised they were looking at a language that could describe the analysis process itself.
- A template which described a “user story: in terms of “As a [X], I want [Y], so that [Z]” was already in use within the company Dan worked for.Whether using UML, little stick figures or swim lanes, user stories can take many forms, but they still had a problem, they do not readily translate to program code as they were.Q. How many have used, or still use user stories?
- The next leap Dan made was to realize that by describing a story’s “behaviour”, simply by rephrasing the user story into “Given, When, Then” statements, meant he had the acceptance criteria there on a platter, as it were, describing the story in terms of scenarios.This seemingly simple change bridged the gap between a product sponsor or product owner and developer, tester or analyst.Some of you may be wondering how is it really any help, or how can it work. Wait a little and you’ll see – initially Shannon and I both had the same thoughts too.
- The benefit of using expressions like “Given When Then” was a non-technical person could think through scenarios, different situations and describe the behaviours of the software in each case.The next benefit Dan was able to see was taking this plain English statement through a small program and generating a test, the difference in this test (against the tests he had been doing before) was this was now an end to end test of a behaviour. Much higher level than the unit tests forming the basis of his Test Driven Development, this test could now replace most (or all) of the usual integration tests he might do, except these tests now had a purpose - to prove a feature, or vertical slice, of the application worked.String together your BDD acceptance tests, and the combination of all of the scenarios described the behaviour of the application.He had a ubiquitous language – something everyone understood.
- With BDD Dan could now generate self documentation, but he did not have to abandon TDD in the process, instead he found a way to make it work better.So back to TDD for a moment. TDD or Test Driven Development, has traditionally been applied to the development process from the developer on. TDD has often focussed on the small detail and while this is not bad, in and of itself, it can cause some catastrophic failures when the big picture gets forgotten. TDD has often left testers out in the cold, whereas BDD involves testers and analysts / BAs discussing acceptance criteria and providing the given - when - then scenarios, and then programmers and testers get together to get the product built and tested. See: http://dannorth.net/2006/06/04/theres-more-to-bdd-than-evolving-tdd/ TDD canbe difficult to “sell” to management – the overheads of test code on top of program code can appear too expensive. Few people, apart from the developers, can read the test code itself; even if your developers try to be open and transparent to testers, analysts, program owners etc., they still need to translate tests in an application and the test results to others.BDD however does not need to be an “either / or” thing .. That is, there’s no reason to stop doing TDD, (and every reason to start TDD if you are not), and there are compelling reasons to include BDD in your current project, in addition to the suite of TDD unit tests.
- Gherkin is the language that Cucumber understands. It is a Business Readable, Domain Specific Language that lets you describe software’s behaviour without detailing how that behaviour is implemented.Gherkin serves two purposes – documentation and automated tests. The third is a bonus feature – when it yells in red it’s talking to you, telling you what code you should write.
- http://www.puppetlabs.com/puppet/introduction/Puppet is an enterprise systems management platform that standardizes the way IT staff deploy and manage infrastructure in the enterprise and the cloud.By automating the provisioning, patching, and configuration of operating system and application components across infrastructure, Puppet enables IT staff to master their infrastructure even as complexity grows. Good support for Linux, Solaris, *nix, early support for Windows Servershttp://www.opscode.com/chefChef is an open-source systems integration framework built specifically for automating the cloud. No matter how complex the realities of your business, Chef makes it easy to deploy servers and scale applications throughout your entire infrastructure. Because it combines the fundamental elements of configuration management and service oriented architectures with the full power of Ruby, Chef makes it easy to create an elegant, fully automated infrastructure.(similar to the old Microsoft System Management Server SMS – now called System Center)
- http://www.puppetlabs.com/puppet/introduction/Puppet is an enterprise systems management platform that standardizes the way IT staff deploy and manage infrastructure in the enterprise and the cloud.By automating the provisioning, patching, and configuration of operating system and application components across infrastructure, Puppet enables IT staff to master their infrastructure even as complexity grows. Good support for Linux, Solaris, *nix, early support for Windows Servershttp://www.opscode.com/chefChef is an open-source systems integration framework built specifically for automating the cloud. No matter how complex the realities of your business, Chef makes it easy to deploy servers and scale applications throughout your entire infrastructure. Because it combines the fundamental elements of configuration management and service oriented architectures with the full power of Ruby, Chef makes it easy to create an elegant, fully automated infrastructure.(similar to the old Microsoft System Management Server SMS – now called System Center)
- Facts can either be set globally as described above or from Given steps which makes them specific to that scenario. It is also possible to load a node’s yaml file and compile a catalog for that specific node.A catalog policy is a list of rules that apply to all catalogs. Above is an example of a simple policy that is distributed with cucumber-puppethttp://projects.puppetlabs.com/projects/cucumber-puppet/wiki
- StoryQ first introduced to us in a presentation by David Star at the North American NDC 2010. In software engineering, a fluent interface (as first coined by Eric Evans and Martin Fowler) is a way of implementing an object oriented API in a way that aims to provide for more readable code.Eg. Example.SetColour(“Red”).SetHeight(2).SetWidth(3);http://en.wikipedia.org/wiki/Fluent_interface
- http://storyq.codeplex.com/
- http://storyq.codeplex.com/Tests also show as pending if not implemented or fail if run-time error or test scenario fails.
- Demonstration of BDD for a web application in Visual Studio 2008 , and in Visual Studio 2010 for a WPF application.
- - Product Owner is well educated on BDD and sees value in the process.- PO provides stories/acceptance criteria in Given/When/Then structureAcceptance Tests created at beginning of sprint. Helps us identify problems early. Raises questions that can be answered by PO before get to implementing the story.Project Sponsor trusts the process because he has seen the resultsBDD/StoryQ reports act as documentation Documentation is living/breathing and automatically gets updated when the code changes