Functional testing the_good_the_bad_and_the_ugly
•
27 j'aime•11,162 vues
In the pursuit of continuous delivery, here is our reality: Unit testing is good, but just not enough. Every Project dedicate a big Functional Testing part at the end to deliver only the top of the testing pyramid. Then the questions are: - How can you deliver quickly if you need to retest your entire application manually for each release? - What automation tools can you use to ensure that your automated tests do not become a maintenance nightmare? - How can you write automated tests that testers can leverage to make their own manual testing more effective? The Agile Sydney Community is proud to receive John Ferguson Smart. Come and join us to challenge John on giving a live demonstration of automated acceptance criteria using JBehave, Selenium 2 and Thucydides. This will give us the opportunity to discuss a number of open source and commercial automated testing tools, with many examples and stories from real-world projects. John talks about a range of tools and approaches to automated functional testing, and discuss the place of functional testing in a modern agile project. He will look at topics such as: - Where does testing fit into an agile development life-cycle - Why it's too late to start testing at the end of the project or iteration - What tests should you automate, and how - What tools work best for different types of tests.
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (20)
Similaire à Functional testing the_good_the_bad_and_the_ugly
Similaire à Functional testing the_good_the_bad_and_the_ugly (20)
Plus de John Ferguson Smart Limited
Plus de John Ferguson Smart Limited (20)
Dernier
Dernier (20)
Functional testing the_good_the_bad_and_the_ugly
- 3. Technical Agile Practices Test Driven Development Continuous Integration/Delivery Automated Acceptance Criteria Refactoring Pair Programming “Without these technical prac1ces, your agile adop1on is hollow” -‐ Ma6 Wynne
- 4. Technical Agile Practices Test Driven Development Continuous Integration/Delivery Automated Acceptance Criteria Refactoring Pair Programming “Without these technical prac1ces, your agile adop1on is hollow” -‐ Ma6 Wynne
- 5. A good func<onal test tells a story
- 7. QA loses its potential if only done at the end of the project
- 9. Living Documentation - test results that serve everyone
- 10. So show me the tools!
- 12. in theory in practice The Record-Replay Scam
- 13. What do ogres and good functional tests have in common?
- 14. (This test is not like an ogre)
- 15. Remember...good automated acceptance tests have layers
- 16. Start off with the business objectives Business-‐level requirements
- 17. Describe the business flow High-‐level steps (business flow)
- 18. Describe the page interactions Page-‐level interac<ons
- 19. Page implementation details HTML details go here
- 20. Business Rules Business Flow Page/Component interac<ons Page/Component details Good automated acceptance tests have layers
- 21. Come on - can we see the tools now?
- 22. Unit tes<ng BDD tools (39%) Given/When/Then (40%) Tabular (13%)
- 23. WebDriver / Selenium 2 Mature browser automa<on Wide adop<on Cross-‐browser support Cross-‐browser support
- 24. ...or use Selenium Hub
- 25. appium Uses the WebDriver API Can run on SauceLabs (with some limita<ons)
- 26. Watir Web Application Testing in Ruby browser.goto 'http://bit.ly/watir-example' browser.text_field(:name => 'entry.0.single').set 'Watir' browser.checkbox(:value => 'Ruby').set Ruby DSL for web tes<ng WebDriver-‐based Fluent and readable No na<ve support for Page Objects?
- 27. Geb Groovy Browser Automation Browser.drive { go "http://myapp.com/login" assert $("h1").text() == "Please Login" $("form.login").with { username = "admin" password = "password" login().click() } assert $("h1").text() == "Admin Section" } Groovy DSL for web tes<ng WebDriver-‐based Fluent and readable Supports Page Objects
- 28. 28 High level BDD Many tools are available
- 29. 29 Narrative: In order to increase sales of advertised articles As a seller I want buyers to be able to easily find ads for articles they want to buy Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of ads for puppies for sale. search_by_keyword_and_location.story Business-‐friendly specifica<ons An example in JBehave
- 30. 30 Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of ads for puppies for sale. search_by_keyword_and_location.story Scenario: Searching by keyword and location Given Sally wants to buy a <present> for her son When she looks for '<present>' in the '<category>' category Then she should obtain a list of ads for <expected> for sale. Examples: |present |category |expected| |puppy |Pets & Animals | puppies| |kitten |Pets & Animals | kittens| |seiko |Jewellery & Watches| watch | Tabular examples
- 31. 31 Scenario: Searching by keyword and location Given Sally wants to buy a puppy for her son When she looks for 'puppy' in the 'Pets and Animals' category Then she should obtain a list of puppy ads search_by_keyword_and_location.story public class SearchAdsSteps { @Steps BuyerSteps buyer; @Given("Sally wants to buy a $present for her son") public void buyingAPresent(String present) { buyer.opens_home_page(); } @When("she looks for $keyword in the $category category") public void adSearchByCategoryAndKeyword(String category, String keyword) { buyer.chooses_category_and_keywords(category, keyword); buyer.performs_search(); } @Then("she should obtain a list of $keyword ads") public void shouldOnlySeeAdsContainingKeyword(String keyword) { buyer.should_only_see_results_with_titles_containing(keyword); } } Step implementa<ons in your favorite language
- 32. 32 From web tests to living documentation
- 34. 34 2 Automate your acceptance criteria 1 Discover your acceptance criteria 4 Execute your acceptance tests 3 Implement your acceptance criteria
- 35. 35 1 Discover your acceptance criteria Feature: Browse Catalog In order to find items that I would like to buy As a customer I want to be able to browse through the catalog Story: Browse by category In order to find items more easily As a customer I want to be able to browse through the product categories Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Define acceptance criteria for each story
- 36. 36 1 Discover your acceptance criteria Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes Clarify the acceptance criteria with examples
- 37. 37 2 Automate your acceptance criteria Story: Browse by category In order to find items more easily As a customer I want to be able to browse through the product categories Acceptance Criteria See all the top-level categories Browse through the category hierarchy Should display the correct products for each category Each category should have the correct sub-categories Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes Narrative: In order to find items more easily As a customer I want to be able to see what product categories exist Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes We now have executable specifica<ons
- 38. 38 2 Automate your acceptance criteria ...but they are reported as ‘pending’
- 39. 39 3 Implement your acceptance criteria Narrative: In order to find items more easily As a customer I want to be able to see what product categories exist Scenario: See all top-level categories Given I want to browse the catalog When I am on the home page Then I should see the following product categories: Clothing, Accessories, Shoes
- 40. 40 4 Execute your acceptance tests What tests have been run
- 41. 41 What requirements have been tested? 4 Execute your acceptance tests
- 42. 42 4 Execute your acceptance tests How did we show that a par<cular requirement worked?
- 43. 43 4 Execute your acceptance tests How did we show that a par<cular requirement worked?
- 44. Demo time!
- 46. John Ferguson Smart Thank you