SlideShare une entreprise Scribd logo

Software Testing on the Web

•
•
Martin Schütte
Martin Schütte

Overview of some software testing tools with focus on PHP and website tests Session at WPCamp 2012 Berlin.

Technologie
1  sur  51
Télécharger pour lire hors ligne
So ware Testing on the Web Martin Schütte
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Outline So ware Quality Documentation Unit Testing PHPUnit Continuity Web Drivers Behaviour Testing Meta PHP Behat and Ruby Cucumber Codeception Martin Schütte So ware Testing on the Web 2012-10-13 2 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Main Questions How can I know (my) so ware is correct? How does my boss know so ware is correct? How can we discuss what “correct” is, anyway? We always need: • implicit assumptions, • explicit specifications. Martin Schütte So ware Testing on the Web 2012-10-13 3 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Walking on water and developing so ware from a specification are easy if both are frozen. – Edward V. Berard Let’s just update PHP… Let’s just change the DB schema… Let’s just add feature X… Is the so ware still correct? Martin Schütte So ware Testing on the Web 2012-10-13 4 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maintainability requires Documentation Consistency of: 1. Code 2. Comments 3. Documentation When code and comments disagree, both are probably wrong. – Norm Schryer Martin Schütte So ware Testing on the Web 2012-10-13 5 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Levels of Documentation . abstract UI External API Subsystems Libraries, Functions concrete Martin Schütte So ware Testing on the Web 2012-10-13 6 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPdoc/JavaDoc/Doxygen etc. /** * Gets single server parameter for specified key. * * @param string $key A key of the parameter to get * @param string $default A default value when key is undefined * * @return string A value of the parameter */ public function getServerParameter($key, $default = ’’) { return (isset($this->server[$key])) ? $this->server[$key] : $default; } Martin Schütte So ware Testing on the Web 2012-10-13 7 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Low Level Documentation Martin Schütte So ware Testing on the Web 2012-10-13 8 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . High Level Documentation Martin Schütte So ware Testing on the Web 2012-10-13 9 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Levels of Testing . abstract Acceptance Tests System Tests Unit Tests concrete Martin Schütte So ware Testing on the Web 2012-10-13 10 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Regression Testing Bug Tracker = Documentation Martin Schütte So ware Testing on the Web 2012-10-13 11 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit class DTABaseTest extends PHPUnit_Framework_TestCase { protected function setUp() { $this->fixture = new DTABase(); } public function testGetNumTooShort() { $input = ”12345”; $off = 3; $len = 4; $rc = $this->fixture->getNum($input, $off, $len); $this->assertEquals(”45”, $rc); } // ... } Martin Schütte So ware Testing on the Web 2012-10-13 12 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit ~/Payment_DTA> phpunit tests/DTABaseTest.php PHPUnit 3.6.11 by Sebastian Bergmann. F.... Time: 0 seconds, Memory: 3.75Mb There was 1 failure: 1) DTABaseTest::testGetNumTooShort Failed asserting that 45.001 matches expected ’45’. /usr/home/mschuett/Payment_DTA/tests/DTABaseTest.php:50 /usr/local/bin/phpunit:46 FAILURES! Tests: 5, Assertions: 5, Failures: 1. Martin Schütte So ware Testing on the Web 2012-10-13 13 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tests are Documentation ~/Payment_DTA> phpunit --testdox tests/DTABaseTest.php PHPUnit 3.6.11 by Sebastian Bergmann. DTABase [ ] Get num too short [x] Get num offset too big [x] Get str too short [x] Get str offset too big [x] Get str invalid Martin Schütte So ware Testing on the Web 2012-10-13 14 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . WordPress Unit Tests . http://unit-tests.trac.wordpress.org/ vagrant@wp:~/trunk$ phpunit Running as single site... To run multisite, use -c multisite.xml Not running ajax tests... To execute these, use --group ajax. PHPUnit 3.6.12 by Sebastian Bergmann. Configuration read from /home/vagrant/trunk/phpunit.xml ................F....................................SS.....S 61 / 1437 ( 4%) S...SSSSSS.............S..SSS..........SS................S... 122 / 1437 ( 8%) .......SS.SSSS...............S.................S............. 183 / 1437 ( 12%) ...............................S............S................ 244 / 1437 ( 16%) .S........................S.................S.......I........ 305 / 1437 ( 21%) .........................SSSS.S..SSS......................... 366 / 1437 ( 25%) ............................................................. 427 / 1437 ( 29%) ............................................................. 488 / 1437 ( 33%) ............................................................. 549 / 1437 ( 38%) ............................................................. 610 / 1437 ( 42%) ..................................................S.......... 671 / 1437 ( 46%) .......SSSSSSSSSSSSSS...........SS........................... 732 / 1437 ( 50%) .............SS...............SSSSSSSSSSSSSSSSSS............. 793 / 1437 ( 55%) ................SSS...S.SSSSSSS................SSSSSSS....... 854 / 1437 ( 59%) .............S..SS....FFF....SSSS.SS......................SSS 915 / 1437 ( 63%) ............................................................. 976 / 1437 ( 67%) ......S..................................................S..S 1037 / 1437 ( 72%) ................SS.S.........................S............... 1098 / 1437 ( 76%) ...............S..................S.......................... 1159 / 1437 ( 80%) ............................................................. 1220 / 1437 ( 84%) .......................F..............................S...... 1281 / 1437 ( 89%) ............................................................. 1342 / 1437 ( 93%) .............................................SS. Time: 08:56, Memory: 122.75Mb [...] FAILURES! Tests: 1390, Assertions: 5640, Failures: 5, Incomplete: 1, Skipped: 117. Martin Schütte So ware Testing on the Web 2012-10-13 15 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit Code Coverage Martin Schütte So ware Testing on the Web 2012-10-13 16 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Things to build upon Automated Testing • git bisect run phpunit: find commit that introduces a bug • Continuous Integration: • frequent check-ins to mainline branch • verified by automated builds and tests • Continuous Deployment: • automated deployment to production • useful: automated rollback Martin Schütte So ware Testing on the Web 2012-10-13 17 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example: Jenkins CI Martin Schütte So ware Testing on the Web 2012-10-13 18 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example: Travis CI for GitHub Martin Schütte So ware Testing on the Web 2012-10-13 19 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Approaches to Unit Testing • Exploratory testing (e. g. for 3rd party libs) • Regression testing (for found & fixed bugs) • Spike and Stabilise testing (a er code, before refactoring) • Test Driven Development (TDD): test first, then code Martin Schütte So ware Testing on the Web 2012-10-13 20 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Program testing can be used to show the presence of bugs, but never to show their absence! – Edsger Dijkstra, Notes on Structured Programming Testing by itself does not improve so ware quality. Test results are an indicator of quality, but in and of themselves, they don’t improve it. […] If you want to improve your so ware, don’t test more; develop better. – Steve McConnell, Code Complete Martin Schütte So ware Testing on the Web 2012-10-13 21 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cost of Testing • Testing (like Documentation) has a cost • o en: productivity improvement > cost • but ROI depends on type of project/so ware Martin Schütte So ware Testing on the Web 2012-10-13 22 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP API class TumblrAPITest extends PHPUnit_Framework_TestCase { public function testPostDatetime() { $url = ”http://staff.tumblr.com/api/read”; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $xmlsrc = curl_exec($ch); $xmlobj = new SimpleXMLElement($xmlsrc); $str_date = date_timestamp_get(date_create( $xmlobj->posts->post[0][’date-gmt’])); $unix_date = intval( $xmlobj->posts->post[0][’unix-timestamp’]); $this->assertEquals($str_date, $unix_date); } } Martin Schütte So ware Testing on the Web 2012-10-13 23 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTML-Content class WpCampTest extends PHPUnit_Framework_TestCase { public function testSiteTitle() { $url = ”http://wpcamp.de/”; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $xmlsrc = curl_exec($ch); $dom = @DOMDocument::loadHTML($xmlsrc); $title = $dom->getElementsByTagName(’title’) ->item(0)->textContent; $this->assertContains(’Das WordPress Event’, $title); } } . Martin Schütte So ware Testing on the Web 2012-10-13 24 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Goutte a simple PHP Web Scraper class WpCampTest extends PHPUnit_Framework_TestCase { public function testSiteTitle() { $client = new Client(); $crawler = $client ->request(’GET’, ’http://wpcamp.de/’); $title = $crawler ->filter(’html head title’) ->first()->text(); $this->assertContains(’Das WordPress Event’, $title); } } Martin Schütte So ware Testing on the Web 2012-10-13 25 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selenium a suite of tools to automate web browsers across platforms class WpTest extends PHPUnit_Extensions_SeleniumTestCase { public function setUp() { $this->setHost(”localhost”); $this->setBrowser(”firefox”); $this->setBrowserUrl(”http://wpcamp.de/”); } public function testSiteTitle() { $this->open(”/”); $this->assertText(”//html/head/title”, ”Das WordPress Event *”); } Martin Schütte So ware Testing on the Web 2012-10-13 26 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selenium, contd. public function testSearch() { $this->open(”/”); $this->type(”id=s”, ”sponsoren”); $this->click(”id=searchsubmit”); $this->waitForPageToLoad(”30000”); $this->assertText(”css=#content”, ”regexp:.*Premium-Sponsor Chocri.*”); $this->assertText(”css=#content”, ”regexp:.*Platin-Sponsor Adobe.*”); } } Martin Schütte So ware Testing on the Web 2012-10-13 27 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Mink an acceptance test framework for web applications class WpCampTest extends PHPUnit_Framework_TestCase { public function setUp() { $driver = new BehatMinkDriverGoutteDriver(); $this->session = new BehatMinkSession($driver); } public function testSiteTitle() { $this->session->visit(’http://wpcamp.de/’); $title = $this->session->getPage() ->find(’css’, ’html head title’)->getText(); $this->assertContains(’Das WordPress Event’, $title); } Martin Schütte So ware Testing on the Web 2012-10-13 28 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Mink, contd. public function testSearch() { $this->session->visit(’http://wpcamp.de/’); $page = $this->session->getPage(); $page->find(’css’, ’#s’)->setValue(’Sponsoren’); $page->find(’css’, ’#searchsubmit’)->press(); $text = $this->session->getPage() ->find(’css’, ’#content’)->getText(); $this->assertContains( ’Premium-Sponsor Chocri’, $text); $this->assertContains( ’Platin-Sponsor Adobe’, $text); } } Martin Schütte So ware Testing on the Web 2012-10-13 29 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture PHPUnit . Mink Selenium Sahi Goutte Zombie.js Firefox Opera Martin Schütte So ware Testing on the Web 2012-10-13 30 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems with Web Drivers Differences in: • web functionality (HTML, CSS, JS, AJAX) ⇒ performance • selectors (ID, CSS, XPath, DOM) • test functionality and integration with test framework Recommendation: Select two Drivers, 1. simple and fast one for unit testing and 2. Selenium/Sahi for integration and UX testing. Martin Schütte So ware Testing on the Web 2012-10-13 31 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Testing in Production = Monitoring Example: check webshop availability Possible layers: • ping server • check HTTP status ”200 OK” • check homepage title • login as user • complete order process Martin Schütte So ware Testing on the Web 2012-10-13 32 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The next level: Behaviour Tests Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen Angenommen ich bin auf ”/” Und ich suche nach ”sponsoren” Dann ich sollte ”Premium-Sponsor Chocri” sehen Und ich sollte ”Platin-Sponsor Adobe” sehen Martin Schütte So ware Testing on the Web 2012-10-13 33 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Next Level: Behaviour Driven Development TDD with other syntax? Is BDD the same as TDD? Yes. If you’re a programmer, and your entire team is programmers, and all your stakeholders are programmers. – Dan North We don’t call them “acceptance tests” because you can’t ask a business person “Please help me with my acceptance test”. Try “I’d like to talk to you about the scenario where…” instead. Or, “Can you give me an example?” Either of these are good. – Liz Keogh Martin Schütte So ware Testing on the Web 2012-10-13 34 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unit & Behaviour Testing Unit Tests Behaviour Tests • unit testing • acceptance test scenarios • programmers • non-developers • programming language • language of business domain • bottom-up • top-down / outside-in • assertXYZ • X should do Y • tests derived from user stories • execute user stories ⇒ development tool ⇒ communication tool Martin Schütte So ware Testing on the Web 2012-10-13 35 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . One Notation/“Language”: Gherkin Feature: [one line describing the story] In order to [benefit] As a [role] I want [feature] Scenario: [A scenario specific goal] Given [Something that needs to have happened or be true] When [Some task I must do] And [Another task] Then [Some way I know I’ve achieved my goal] Martin Schütte So ware Testing on the Web 2012-10-13 36 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I18N # language: de Funktionalität: [one line describing the story] Um zu erreichen [benefit] Als [role] Möchte ich [feature] Szenario: [Title] Angenommen [context] Wenn [event] Dann [outcome] Und [another outcome] Martin Schütte So ware Testing on the Web 2012-10-13 37 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TDD with other syntax? Scenario: New lists are empty Given a new list Then the list should be empty. Scenario: Lists with things in them are not empty. Given a new list When we add an object Then the list should not be empty. . Martin Schütte So ware Testing on the Web 2012-10-13 38 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implementation / Translation A look behind the curtain: • framework is clever but not magical • some translation needed • statements have to become executable code Mechanism: • plain sentence → method name • quoted words → arguments • matching with annotated RegExp • methods yield success, exception, or pending exception Martin Schütte So ware Testing on the Web 2012-10-13 39 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat/Mink FeatureContext.php use BehatMinkExtensionContextMinkContext; class FeatureContext extends BehatContext { public function __construct(array $parameters) { $this->useContext(’mink’, new MinkContext()); } } MinkContext adds web-related methods and their I18N translations, e. g.: • visit($page) • fillField($field, $value) • pressButton($button) Martin Schütte So ware Testing on the Web 2012-10-13 40 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Output [mschuett@dagny] ~/wpcamp/code% ./vendor/bin/behat Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen # features/wpcamp.feature:8 Angenommen ich bin auf ”/” # BehatMinkExtensionContextMinkContext::visit Und ich suche nach ”sponsoren” Dann ich sollte ”Premium-Sponsor Chocri” sehen # BehatMinkExtensionContextMinkContext::asser Und ich sollte ”Platin-Sponsor Adobe” sehen # BehatMinkExtensionContextMinkContext::asser 1 scenario (1 undefined) 4 steps (1 passed, 2 skipped, 1 undefined) 0m1.574s You can implement step definitions for undefined steps with these snippets: /** * @Given /^ich suche nach ”([^”]*)”$/ */ public function ichSucheNach($arg1) { throw new PendingException(); } Martin Schütte So ware Testing on the Web 2012-10-13 41 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Edit FeatureContext.php add method /** * @Given /^ich suche nach ”([^”]*)”$/ */ public function ichSucheNach($arg) { return array( new When(”I fill in ”s” with ”$arg””), new When(”I press ”searchsubmit””), ); } Martin Schütte So ware Testing on the Web 2012-10-13 42 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Output [mschuett@dagny] ~/wpcamp/code% ./vendor/bin/behat Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen # features/wpcamp.feature:8 Angenommen ich bin auf ”/” # BehatMinkExtensionContextMinkContext::visit Und ich suche nach ”sponsoren” # FeatureContext::ichSucheNach() Dann ich sollte ”Premium-Sponsor Chocri” sehen # BehatMinkExtensionContextMinkContext::asser Und ich sollte ”Platin-Sponsor Adobe” sehen # BehatMinkExtensionContextMinkContext::asser 1 scenario (1 passed) 4 steps (4 passed) 0m2.59s Martin Schütte So ware Testing on the Web 2012-10-13 43 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture (PHP) Behat PHPUnit . Mink Selenium Sahi Goutte Zombie.js Firefox Opera Martin Schütte So ware Testing on the Web 2012-10-13 44 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cucumber original Ruby version, nicer syntax # Example Webrat step definitions When /I search for ”(.*)”/ do |query| @browser.text_field(:name, ’s’).set(query) @browser.button(:name, ’searchsubmit’).click end Then /I should see/ do |text| @browser.text.should =~ /#{text}/m end Martin Schütte So ware Testing on the Web 2012-10-13 45 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture (Ruby) . Cucumber Selenium Watir Webrat Celerity Firefox Opera Martin Schütte So ware Testing on the Web 2012-10-13 46 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception unit test a new full-stack testing PHP framework $I = new CodeGuy($scenario); $I->wantTo(’perform actions and see results’); $I->testMethod(’DateTime.getTimestamp’); $date = new DateTime(”2012-10-13T14:45:00+02:00”); $I->executeTestedMethodOn($date); $I->seeResultIs(’int’); $I->seeResultEquals(1350132300); Martin Schütte So ware Testing on the Web 2012-10-13 47 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception acceptance test $I = new WebGuy($scenario); $I->wantTo(’find infos on the website’); $I->see(’Das WordPress Event’, ’html head title’); $I->fillField(’s’, ’sponsoren’); $I->click(’searchsubmit’); $I->see(’Premium-Sponsor Chocri’, ’#content’); $I->see(’Platin-Sponsor Adobe’, ’#content’); Martin Schütte So ware Testing on the Web 2012-10-13 48 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception Output [mschuett@dagny] ~/wpcamp/code% php codecept.phar run --steps Codeception PHP Testing Framework v1.1.4 Powered by PHPUnit 3.6.10 by Sebastian Bergmann. Suite unit started Trying to perform actions and see results (exampleCept.php) Scenario: * I test method ”DateTime.getTimestamp” * I execute tested method on ”DateTime” * I see result is ”int” * I see result equals ”1350132300” OK Suite functional started Suite acceptance started Trying to find infos on the website (wpcampCept.php) Scenario: * I am on page ”/” * I see ”Das WordPress Event”,”html head title” * I fill field ”s”,”sponsoren” * I click ”searchsubmit” * I see ”Premium-Sponsor Chocri”,”#content” * I see ”Platin-Sponsor Adobe”,”#content” OK Time: 4 seconds, Memory: 9.50Mb OK (2 tests, 5 assertions) Martin Schütte So ware Testing on the Web 2012-10-13 49 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems • most Web tests require known DB state • behaviour testing is slow • only run locally • use in-memory DB • draw the line • features in documentation (what) → add behaviour test • mere implementation (how) → only unit tests Martin Schütte So ware Testing on the Web 2012-10-13 50 / 51
So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary Value individuals and interactions over processes and tools – The Agile Manifesto Links: • PHPUnit • Sebastian Bergmann (PHPUnit) • Selenium • Dan North’s blog (BDD) • Behat & Mink • Liz Keogh’s blog (BDD) • Cucumber • Geek and Poke • Codeception • Not Invented Here Martin Schütte So ware Testing on the Web 2012-10-13 51 / 51

Recommandé

Code Conventions
Code ConventionsCode Conventions
Code Conventions51 lecture
 
Rfb
RfbRfb
RfbSampath Shetty
 
Shariar Rostami - Master Thesis
Shariar Rostami - Master ThesisShariar Rostami - Master Thesis
Shariar Rostami - Master Thesisshahriar-ro
 
Climb - A Generic and Dynamic Approach to Image Processing
Climb - A Generic and Dynamic Approach to Image ProcessingClimb - A Generic and Dynamic Approach to Image Processing
Climb - A Generic and Dynamic Approach to Image ProcessingChristopher Chedeau
 
Asme v 2001 article 6
Asme v 2001 article 6Asme v 2001 article 6
Asme v 2001 article 6Subramaniam Ramasamy
 
Installing sql server 2012 failover cluster instance
Installing sql server 2012 failover cluster instanceInstalling sql server 2012 failover cluster instance
Installing sql server 2012 failover cluster instanceDavid Muise
 
Bufferauthentication
BufferauthenticationBufferauthentication
BufferauthenticationVishal Shah
 
Location logic working_with_content
Location logic working_with_contentLocation logic working_with_content
Location logic working_with_contentsorenk22
 

Recommandé

Code Conventions
Code ConventionsCode Conventions
Code Conventions51 lecture
 
Rfb
RfbRfb
RfbSampath Shetty
 
Shariar Rostami - Master Thesis
Shariar Rostami - Master ThesisShariar Rostami - Master Thesis
Shariar Rostami - Master Thesisshahriar-ro
 
Climb - A Generic and Dynamic Approach to Image Processing
Climb - A Generic and Dynamic Approach to Image ProcessingClimb - A Generic and Dynamic Approach to Image Processing
Climb - A Generic and Dynamic Approach to Image ProcessingChristopher Chedeau
 
Asme v 2001 article 6
Asme v 2001 article 6Asme v 2001 article 6
Asme v 2001 article 6Subramaniam Ramasamy
 
Installing sql server 2012 failover cluster instance
Installing sql server 2012 failover cluster instanceInstalling sql server 2012 failover cluster instance
Installing sql server 2012 failover cluster instanceDavid Muise
 
Bufferauthentication
BufferauthenticationBufferauthentication
BufferauthenticationVishal Shah
 
Location logic working_with_content
Location logic working_with_contentLocation logic working_with_content
Location logic working_with_contentsorenk22
 
DuPont Krytox Product Overview, H 58505 5
DuPont Krytox Product Overview, H 58505 5DuPont Krytox Product Overview, H 58505 5
DuPont Krytox Product Overview, H 58505 5Jim_Bradshaw
 
Gates Belts PM Manual
Gates Belts PM ManualGates Belts PM Manual
Gates Belts PM ManualRicky Smith CMRP, CMRT
 
Zrp manual 2
Zrp manual 2Zrp manual 2
Zrp manual 2Ronald Hutabarat
 
test
testtest
testsprinklrdev
 
Gdb
GdbGdb
GdbShantanu Sharma, Ph.D.
 
US High Performance Components Brochure
US High Performance Components BrochureUS High Performance Components Brochure
US High Performance Components BrochureHuntsman Advanced Materials US
 
Java web programming
Java web programmingJava web programming
Java web programmingMumbai Academisc
 
Ssl2
Ssl2Ssl2
Ssl2karthickmsit
 
Mixed Streaming of Video over Wireless Networks
Mixed Streaming of Video over Wireless NetworksMixed Streaming of Video over Wireless Networks
Mixed Streaming of Video over Wireless NetworksVideoguy
 
PostgreSQL 14 Beta 1 New Features with Examples (English)
PostgreSQL 14 Beta 1 New Features with Examples (English)PostgreSQL 14 Beta 1 New Features with Examples (English)
PostgreSQL 14 Beta 1 New Features with Examples (English)Noriyoshi Shinoda
 
Program Directory for IBM Ported Tools for z/OS
Program Directory for IBM Ported Tools for z/OSProgram Directory for IBM Ported Tools for z/OS
Program Directory for IBM Ported Tools for z/OSIBM India Smarter Computing
 
The Dissertation
The DissertationThe Dissertation
The Dissertationphooji
 
Pspice userguide ingles
Pspice userguide inglesPspice userguide ingles
Pspice userguide inglesunoenero
 
test5
test5test5
test5Qingxiu Chen
 
Sdd 2
Sdd 2Sdd 2
Sdd 2Auro Infomation Technology
 
test6
test6test6
test6Qingxiu Chen
 
test4
test4test4
test4Qingxiu Chen
 
test5
test5test5
test5Qingxiu Chen
 
test6
test6test6
test6Qingxiu Chen
 
Certification guide series ibm tivoli provisioning manager express for softwa...
Certification guide series ibm tivoli provisioning manager express for softwa...Certification guide series ibm tivoli provisioning manager express for softwa...
Certification guide series ibm tivoli provisioning manager express for softwa...Banking at Ho Chi Minh city
 
Managing disk subsystems using ibm total storage productivity center sg247097
Managing disk subsystems using ibm total storage productivity center sg247097Managing disk subsystems using ibm total storage productivity center sg247097
Managing disk subsystems using ibm total storage productivity center sg247097Banking at Ho Chi Minh city
 
Is everything ok now?
Is everything ok now?Is everything ok now?
Is everything ok now?DreamMalar
 

Contenu connexe

Tendances

DuPont Krytox Product Overview, H 58505 5
DuPont Krytox Product Overview, H 58505 5DuPont Krytox Product Overview, H 58505 5
DuPont Krytox Product Overview, H 58505 5Jim_Bradshaw
 
Mixed Streaming of Video over Wireless Networks
Mixed Streaming of Video over Wireless NetworksMixed Streaming of Video over Wireless Networks
Mixed Streaming of Video over Wireless NetworksVideoguy
 
PostgreSQL 14 Beta 1 New Features with Examples (English)
PostgreSQL 14 Beta 1 New Features with Examples (English)PostgreSQL 14 Beta 1 New Features with Examples (English)
PostgreSQL 14 Beta 1 New Features with Examples (English)Noriyoshi Shinoda
 
The Dissertation
The DissertationThe Dissertation
The Dissertationphooji
 

Tendances (12)

DuPont Krytox Product Overview, H 58505 5
DuPont Krytox Product Overview, H 58505 5DuPont Krytox Product Overview, H 58505 5
DuPont Krytox Product Overview, H 58505 5
 
Gates Belts PM Manual
Gates Belts PM ManualGates Belts PM Manual
Gates Belts PM Manual
 
Zrp manual 2
Zrp manual 2Zrp manual 2
Zrp manual 2
 
test
testtest
test
 
Gdb
GdbGdb
Gdb
 
US High Performance Components Brochure
US High Performance Components BrochureUS High Performance Components Brochure
US High Performance Components Brochure
 
Java web programming
Java web programmingJava web programming
Java web programming
 
Ssl2
Ssl2Ssl2
Ssl2
 
Mixed Streaming of Video over Wireless Networks
Mixed Streaming of Video over Wireless NetworksMixed Streaming of Video over Wireless Networks
Mixed Streaming of Video over Wireless Networks
 
PostgreSQL 14 Beta 1 New Features with Examples (English)
PostgreSQL 14 Beta 1 New Features with Examples (English)PostgreSQL 14 Beta 1 New Features with Examples (English)
PostgreSQL 14 Beta 1 New Features with Examples (English)
 
Program Directory for IBM Ported Tools for z/OS
Program Directory for IBM Ported Tools for z/OSProgram Directory for IBM Ported Tools for z/OS
Program Directory for IBM Ported Tools for z/OS
 
The Dissertation
The DissertationThe Dissertation
The Dissertation
 

Similaire à Software Testing on the Web

Pspice userguide ingles
Pspice userguide inglesPspice userguide ingles
Pspice userguide inglesunoenero
 
Certification guide series ibm tivoli provisioning manager express for softwa...
Certification guide series ibm tivoli provisioning manager express for softwa...Certification guide series ibm tivoli provisioning manager express for softwa...
Certification guide series ibm tivoli provisioning manager express for softwa...Banking at Ho Chi Minh city
 
Managing disk subsystems using ibm total storage productivity center sg247097
Managing disk subsystems using ibm total storage productivity center sg247097Managing disk subsystems using ibm total storage productivity center sg247097
Managing disk subsystems using ibm total storage productivity center sg247097Banking at Ho Chi Minh city
 
Is everything ok now?
Is everything ok now?Is everything ok now?
Is everything ok now?DreamMalar
 
This is the test for OB Properties
This is the test for OB PropertiesThis is the test for OB Properties
This is the test for OB PropertiesDreamMalar
 
Certification guide series ibm tivoli netcool omn ibus v7.2 implementation sg...
Certification guide series ibm tivoli netcool omn ibus v7.2 implementation sg...Certification guide series ibm tivoli netcool omn ibus v7.2 implementation sg...
Certification guide series ibm tivoli netcool omn ibus v7.2 implementation sg...Banking at Ho Chi Minh city
 
100PercentPureJavaCookbook-4_1_1
100PercentPureJavaCookbook-4_1_1100PercentPureJavaCookbook-4_1_1
100PercentPureJavaCookbook-4_1_1AbrarMoiz
 
DBMS_Lab_Manual_&_Solution
DBMS_Lab_Manual_&_SolutionDBMS_Lab_Manual_&_Solution
DBMS_Lab_Manual_&_SolutionSyed Zaid Irshad
 
Ad Ch.1 8 (1)
Ad Ch.1 8 (1)Ad Ch.1 8 (1)
Ad Ch.1 8 (1)gopi1985
 
XAdES Specification based on the Apache XMLSec Project
XAdES Specification based on the Apache XMLSec Project XAdES Specification based on the Apache XMLSec Project
XAdES Specification based on the Apache XMLSec Project Krassen Deltchev
 
Tivoli and web sphere application server on z os sg247062
Tivoli and web sphere application server on z os sg247062Tivoli and web sphere application server on z os sg247062
Tivoli and web sphere application server on z os sg247062Banking at Ho Chi Minh city
 

Similaire à Software Testing on the Web (20)

Pspice userguide ingles
Pspice userguide inglesPspice userguide ingles
Pspice userguide ingles
 
test5
test5test5
test5
 
Sdd 2
Sdd 2Sdd 2
Sdd 2
 
test6
test6test6
test6
 
test4
test4test4
test4
 
test5
test5test5
test5
 
test6
test6test6
test6
 
Certification guide series ibm tivoli provisioning manager express for softwa...
Certification guide series ibm tivoli provisioning manager express for softwa...Certification guide series ibm tivoli provisioning manager express for softwa...
Certification guide series ibm tivoli provisioning manager express for softwa...
 
Managing disk subsystems using ibm total storage productivity center sg247097
Managing disk subsystems using ibm total storage productivity center sg247097Managing disk subsystems using ibm total storage productivity center sg247097
Managing disk subsystems using ibm total storage productivity center sg247097
 
Is everything ok now?
Is everything ok now?Is everything ok now?
Is everything ok now?
 
This is the test for OB Properties
This is the test for OB PropertiesThis is the test for OB Properties
This is the test for OB Properties
 
Certification guide series ibm tivoli netcool omn ibus v7.2 implementation sg...
Certification guide series ibm tivoli netcool omn ibus v7.2 implementation sg...Certification guide series ibm tivoli netcool omn ibus v7.2 implementation sg...
Certification guide series ibm tivoli netcool omn ibus v7.2 implementation sg...
 
100PercentPureJavaCookbook-4_1_1
100PercentPureJavaCookbook-4_1_1100PercentPureJavaCookbook-4_1_1
100PercentPureJavaCookbook-4_1_1
 
DBMS_Lab_Manual_&_Solution
DBMS_Lab_Manual_&_SolutionDBMS_Lab_Manual_&_Solution
DBMS_Lab_Manual_&_Solution
 
Benchmarks for Digital Preservation tools. Kresimir Duretec, Artur Kulmukhame...
Benchmarks for Digital Preservation tools. Kresimir Duretec, Artur Kulmukhame...Benchmarks for Digital Preservation tools. Kresimir Duretec, Artur Kulmukhame...
Benchmarks for Digital Preservation tools. Kresimir Duretec, Artur Kulmukhame...
 
Ad Ch.1 8 (1)
Ad Ch.1 8 (1)Ad Ch.1 8 (1)
Ad Ch.1 8 (1)
 
Jdbc
JdbcJdbc
Jdbc
 
Moss2007
Moss2007Moss2007
Moss2007
 
XAdES Specification based on the Apache XMLSec Project
XAdES Specification based on the Apache XMLSec Project XAdES Specification based on the Apache XMLSec Project
XAdES Specification based on the Apache XMLSec Project
 
Tivoli and web sphere application server on z os sg247062
Tivoli and web sphere application server on z os sg247062Tivoli and web sphere application server on z os sg247062
Tivoli and web sphere application server on z os sg247062
 

Plus de Martin Schütte

Writing Ansible Modules (DENOG11)
Writing Ansible Modules (DENOG11)Writing Ansible Modules (DENOG11)
Writing Ansible Modules (DENOG11)Martin Schütte
 
Writing Ansible Modules (CLT'19)
Writing Ansible Modules (CLT'19)Writing Ansible Modules (CLT'19)
Writing Ansible Modules (CLT'19)Martin Schütte
 
Terraform – Infrastructure as Code (Kielux'18)
Terraform – Infrastructure as Code (Kielux'18)Terraform – Infrastructure as Code (Kielux'18)
Terraform – Infrastructure as Code (Kielux'18)Martin Schütte
 
Terraform -- Infrastructure as Code
Terraform -- Infrastructure as CodeTerraform -- Infrastructure as Code
Terraform -- Infrastructure as CodeMartin Schütte
 
Terraform: Cloud Configuration Management (WTC/IPC'16)
Terraform: Cloud Configuration Management (WTC/IPC'16)Terraform: Cloud Configuration Management (WTC/IPC'16)
Terraform: Cloud Configuration Management (WTC/IPC'16)Martin Schütte
 
Terraform: Configuration Management for Cloud Services
Terraform: Configuration Management for Cloud ServicesTerraform: Configuration Management for Cloud Services
Terraform: Configuration Management for Cloud ServicesMartin Schütte
 
The IPv6 Snort Plugin (at DeepSec 2014)
The IPv6 Snort Plugin (at DeepSec 2014)The IPv6 Snort Plugin (at DeepSec 2014)
The IPv6 Snort Plugin (at DeepSec 2014)Martin Schütte
 
The IPv6 Snort Plugin (at Troopers 14 IPv6 Security Summit)
The IPv6 Snort Plugin (at Troopers 14 IPv6 Security Summit)The IPv6 Snort Plugin (at Troopers 14 IPv6 Security Summit)
The IPv6 Snort Plugin (at Troopers 14 IPv6 Security Summit)Martin Schütte
 
Short Introduction to IPv6
Short Introduction to IPv6Short Introduction to IPv6
Short Introduction to IPv6Martin Schütte
 
NetBSD syslogd with IETF Syslog Protocols
NetBSD syslogd with IETF Syslog ProtocolsNetBSD syslogd with IETF Syslog Protocols
NetBSD syslogd with IETF Syslog ProtocolsMartin Schütte
 
Design and Implementation of an IPv6 Plugin for the Snort Intrusion Detection...
Design and Implementation of an IPv6 Plugin for the Snort Intrusion Detection...Design and Implementation of an IPv6 Plugin for the Snort Intrusion Detection...
Design and Implementation of an IPv6 Plugin for the Snort Intrusion Detection...Martin Schütte
 

Plus de Martin Schütte (13)

Writing Ansible Modules (DENOG11)
Writing Ansible Modules (DENOG11)Writing Ansible Modules (DENOG11)
Writing Ansible Modules (DENOG11)
 
Writing Ansible Modules (CLT'19)
Writing Ansible Modules (CLT'19)Writing Ansible Modules (CLT'19)
Writing Ansible Modules (CLT'19)
 
Terraform – Infrastructure as Code (Kielux'18)
Terraform – Infrastructure as Code (Kielux'18)Terraform – Infrastructure as Code (Kielux'18)
Terraform – Infrastructure as Code (Kielux'18)
 
Terraform -- Infrastructure as Code
Terraform -- Infrastructure as CodeTerraform -- Infrastructure as Code
Terraform -- Infrastructure as Code
 
Terraform: Cloud Configuration Management (WTC/IPC'16)
Terraform: Cloud Configuration Management (WTC/IPC'16)Terraform: Cloud Configuration Management (WTC/IPC'16)
Terraform: Cloud Configuration Management (WTC/IPC'16)
 
Terraform: Configuration Management for Cloud Services
Terraform: Configuration Management for Cloud ServicesTerraform: Configuration Management for Cloud Services
Terraform: Configuration Management for Cloud Services
 
The IPv6 Snort Plugin (at DeepSec 2014)
The IPv6 Snort Plugin (at DeepSec 2014)The IPv6 Snort Plugin (at DeepSec 2014)
The IPv6 Snort Plugin (at DeepSec 2014)
 
The IPv6 Snort Plugin (at Troopers 14 IPv6 Security Summit)
The IPv6 Snort Plugin (at Troopers 14 IPv6 Security Summit)The IPv6 Snort Plugin (at Troopers 14 IPv6 Security Summit)
The IPv6 Snort Plugin (at Troopers 14 IPv6 Security Summit)
 
Short Introduction to IPv6
Short Introduction to IPv6Short Introduction to IPv6
Short Introduction to IPv6
 
NetBSD syslogd with IETF Syslog Protocols
NetBSD syslogd with IETF Syslog ProtocolsNetBSD syslogd with IETF Syslog Protocols
NetBSD syslogd with IETF Syslog Protocols
 
PGP/GPG Einführung
PGP/GPG EinführungPGP/GPG Einführung
PGP/GPG Einführung
 
Design and Implementation of an IPv6 Plugin for the Snort Intrusion Detection...
Design and Implementation of an IPv6 Plugin for the Snort Intrusion Detection...Design and Implementation of an IPv6 Plugin for the Snort Intrusion Detection...
Design and Implementation of an IPv6 Plugin for the Snort Intrusion Detection...
 
Syslog Protocols
Syslog ProtocolsSyslog Protocols
Syslog Protocols
 

Dernier

Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 

Dernier (20)

Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 

Software Testing on the Web

  • 1. So ware Testing on the Web Martin Schütte
  • 2. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Outline So ware Quality Documentation Unit Testing PHPUnit Continuity Web Drivers Behaviour Testing Meta PHP Behat and Ruby Cucumber Codeception Martin Schütte So ware Testing on the Web 2012-10-13 2 / 51
  • 3. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Main Questions How can I know (my) so ware is correct? How does my boss know so ware is correct? How can we discuss what “correct” is, anyway? We always need: • implicit assumptions, • explicit specifications. Martin Schütte So ware Testing on the Web 2012-10-13 3 / 51
  • 4. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Walking on water and developing so ware from a specification are easy if both are frozen. – Edward V. Berard Let’s just update PHP… Let’s just change the DB schema… Let’s just add feature X… Is the so ware still correct? Martin Schütte So ware Testing on the Web 2012-10-13 4 / 51
  • 5. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maintainability requires Documentation Consistency of: 1. Code 2. Comments 3. Documentation When code and comments disagree, both are probably wrong. – Norm Schryer Martin Schütte So ware Testing on the Web 2012-10-13 5 / 51
  • 6. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Levels of Documentation . abstract UI External API Subsystems Libraries, Functions concrete Martin Schütte So ware Testing on the Web 2012-10-13 6 / 51
  • 7. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPdoc/JavaDoc/Doxygen etc. /** * Gets single server parameter for specified key. * * @param string $key A key of the parameter to get * @param string $default A default value when key is undefined * * @return string A value of the parameter */ public function getServerParameter($key, $default = ’’) { return (isset($this->server[$key])) ? $this->server[$key] : $default; } Martin Schütte So ware Testing on the Web 2012-10-13 7 / 51
  • 8. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Low Level Documentation Martin Schütte So ware Testing on the Web 2012-10-13 8 / 51
  • 9. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . High Level Documentation Martin Schütte So ware Testing on the Web 2012-10-13 9 / 51
  • 10. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Levels of Testing . abstract Acceptance Tests System Tests Unit Tests concrete Martin Schütte So ware Testing on the Web 2012-10-13 10 / 51
  • 11. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Regression Testing Bug Tracker = Documentation Martin Schütte So ware Testing on the Web 2012-10-13 11 / 51
  • 12. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit class DTABaseTest extends PHPUnit_Framework_TestCase { protected function setUp() { $this->fixture = new DTABase(); } public function testGetNumTooShort() { $input = ”12345”; $off = 3; $len = 4; $rc = $this->fixture->getNum($input, $off, $len); $this->assertEquals(”45”, $rc); } // ... } Martin Schütte So ware Testing on the Web 2012-10-13 12 / 51
  • 13. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit ~/Payment_DTA> phpunit tests/DTABaseTest.php PHPUnit 3.6.11 by Sebastian Bergmann. F.... Time: 0 seconds, Memory: 3.75Mb There was 1 failure: 1) DTABaseTest::testGetNumTooShort Failed asserting that 45.001 matches expected ’45’. /usr/home/mschuett/Payment_DTA/tests/DTABaseTest.php:50 /usr/local/bin/phpunit:46 FAILURES! Tests: 5, Assertions: 5, Failures: 1. Martin Schütte So ware Testing on the Web 2012-10-13 13 / 51
  • 14. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tests are Documentation ~/Payment_DTA> phpunit --testdox tests/DTABaseTest.php PHPUnit 3.6.11 by Sebastian Bergmann. DTABase [ ] Get num too short [x] Get num offset too big [x] Get str too short [x] Get str offset too big [x] Get str invalid Martin Schütte So ware Testing on the Web 2012-10-13 14 / 51
  • 15. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . WordPress Unit Tests . http://unit-tests.trac.wordpress.org/ vagrant@wp:~/trunk$ phpunit Running as single site... To run multisite, use -c multisite.xml Not running ajax tests... To execute these, use --group ajax. PHPUnit 3.6.12 by Sebastian Bergmann. Configuration read from /home/vagrant/trunk/phpunit.xml ................F....................................SS.....S 61 / 1437 ( 4%) S...SSSSSS.............S..SSS..........SS................S... 122 / 1437 ( 8%) .......SS.SSSS...............S.................S............. 183 / 1437 ( 12%) ...............................S............S................ 244 / 1437 ( 16%) .S........................S.................S.......I........ 305 / 1437 ( 21%) .........................SSSS.S..SSS......................... 366 / 1437 ( 25%) ............................................................. 427 / 1437 ( 29%) ............................................................. 488 / 1437 ( 33%) ............................................................. 549 / 1437 ( 38%) ............................................................. 610 / 1437 ( 42%) ..................................................S.......... 671 / 1437 ( 46%) .......SSSSSSSSSSSSSS...........SS........................... 732 / 1437 ( 50%) .............SS...............SSSSSSSSSSSSSSSSSS............. 793 / 1437 ( 55%) ................SSS...S.SSSSSSS................SSSSSSS....... 854 / 1437 ( 59%) .............S..SS....FFF....SSSS.SS......................SSS 915 / 1437 ( 63%) ............................................................. 976 / 1437 ( 67%) ......S..................................................S..S 1037 / 1437 ( 72%) ................SS.S.........................S............... 1098 / 1437 ( 76%) ...............S..................S.......................... 1159 / 1437 ( 80%) ............................................................. 1220 / 1437 ( 84%) .......................F..............................S...... 1281 / 1437 ( 89%) ............................................................. 1342 / 1437 ( 93%) .............................................SS. Time: 08:56, Memory: 122.75Mb [...] FAILURES! Tests: 1390, Assertions: 5640, Failures: 5, Incomplete: 1, Skipped: 117. Martin Schütte So ware Testing on the Web 2012-10-13 15 / 51
  • 16. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit Code Coverage Martin Schütte So ware Testing on the Web 2012-10-13 16 / 51
  • 17. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Things to build upon Automated Testing • git bisect run phpunit: find commit that introduces a bug • Continuous Integration: • frequent check-ins to mainline branch • verified by automated builds and tests • Continuous Deployment: • automated deployment to production • useful: automated rollback Martin Schütte So ware Testing on the Web 2012-10-13 17 / 51
  • 18. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example: Jenkins CI Martin Schütte So ware Testing on the Web 2012-10-13 18 / 51
  • 19. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example: Travis CI for GitHub Martin Schütte So ware Testing on the Web 2012-10-13 19 / 51
  • 20. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Approaches to Unit Testing • Exploratory testing (e. g. for 3rd party libs) • Regression testing (for found & fixed bugs) • Spike and Stabilise testing (a er code, before refactoring) • Test Driven Development (TDD): test first, then code Martin Schütte So ware Testing on the Web 2012-10-13 20 / 51
  • 21. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Program testing can be used to show the presence of bugs, but never to show their absence! – Edsger Dijkstra, Notes on Structured Programming Testing by itself does not improve so ware quality. Test results are an indicator of quality, but in and of themselves, they don’t improve it. […] If you want to improve your so ware, don’t test more; develop better. – Steve McConnell, Code Complete Martin Schütte So ware Testing on the Web 2012-10-13 21 / 51
  • 22. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cost of Testing • Testing (like Documentation) has a cost • o en: productivity improvement > cost • but ROI depends on type of project/so ware Martin Schütte So ware Testing on the Web 2012-10-13 22 / 51
  • 23. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP API class TumblrAPITest extends PHPUnit_Framework_TestCase { public function testPostDatetime() { $url = ”http://staff.tumblr.com/api/read”; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $xmlsrc = curl_exec($ch); $xmlobj = new SimpleXMLElement($xmlsrc); $str_date = date_timestamp_get(date_create( $xmlobj->posts->post[0][’date-gmt’])); $unix_date = intval( $xmlobj->posts->post[0][’unix-timestamp’]); $this->assertEquals($str_date, $unix_date); } } Martin Schütte So ware Testing on the Web 2012-10-13 23 / 51
  • 24. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTML-Content class WpCampTest extends PHPUnit_Framework_TestCase { public function testSiteTitle() { $url = ”http://wpcamp.de/”; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $xmlsrc = curl_exec($ch); $dom = @DOMDocument::loadHTML($xmlsrc); $title = $dom->getElementsByTagName(’title’) ->item(0)->textContent; $this->assertContains(’Das WordPress Event’, $title); } } . Martin Schütte So ware Testing on the Web 2012-10-13 24 / 51
  • 25. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Goutte a simple PHP Web Scraper class WpCampTest extends PHPUnit_Framework_TestCase { public function testSiteTitle() { $client = new Client(); $crawler = $client ->request(’GET’, ’http://wpcamp.de/’); $title = $crawler ->filter(’html head title’) ->first()->text(); $this->assertContains(’Das WordPress Event’, $title); } } Martin Schütte So ware Testing on the Web 2012-10-13 25 / 51
  • 26. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selenium a suite of tools to automate web browsers across platforms class WpTest extends PHPUnit_Extensions_SeleniumTestCase { public function setUp() { $this->setHost(”localhost”); $this->setBrowser(”firefox”); $this->setBrowserUrl(”http://wpcamp.de/”); } public function testSiteTitle() { $this->open(”/”); $this->assertText(”//html/head/title”, ”Das WordPress Event *”); } Martin Schütte So ware Testing on the Web 2012-10-13 26 / 51
  • 27. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selenium, contd. public function testSearch() { $this->open(”/”); $this->type(”id=s”, ”sponsoren”); $this->click(”id=searchsubmit”); $this->waitForPageToLoad(”30000”); $this->assertText(”css=#content”, ”regexp:.*Premium-Sponsor Chocri.*”); $this->assertText(”css=#content”, ”regexp:.*Platin-Sponsor Adobe.*”); } } Martin Schütte So ware Testing on the Web 2012-10-13 27 / 51
  • 28. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Mink an acceptance test framework for web applications class WpCampTest extends PHPUnit_Framework_TestCase { public function setUp() { $driver = new BehatMinkDriverGoutteDriver(); $this->session = new BehatMinkSession($driver); } public function testSiteTitle() { $this->session->visit(’http://wpcamp.de/’); $title = $this->session->getPage() ->find(’css’, ’html head title’)->getText(); $this->assertContains(’Das WordPress Event’, $title); } Martin Schütte So ware Testing on the Web 2012-10-13 28 / 51
  • 29. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Mink, contd. public function testSearch() { $this->session->visit(’http://wpcamp.de/’); $page = $this->session->getPage(); $page->find(’css’, ’#s’)->setValue(’Sponsoren’); $page->find(’css’, ’#searchsubmit’)->press(); $text = $this->session->getPage() ->find(’css’, ’#content’)->getText(); $this->assertContains( ’Premium-Sponsor Chocri’, $text); $this->assertContains( ’Platin-Sponsor Adobe’, $text); } } Martin Schütte So ware Testing on the Web 2012-10-13 29 / 51
  • 30. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture PHPUnit . Mink Selenium Sahi Goutte Zombie.js Firefox Opera Martin Schütte So ware Testing on the Web 2012-10-13 30 / 51
  • 31. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems with Web Drivers Differences in: • web functionality (HTML, CSS, JS, AJAX) ⇒ performance • selectors (ID, CSS, XPath, DOM) • test functionality and integration with test framework Recommendation: Select two Drivers, 1. simple and fast one for unit testing and 2. Selenium/Sahi for integration and UX testing. Martin Schütte So ware Testing on the Web 2012-10-13 31 / 51
  • 32. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Testing in Production = Monitoring Example: check webshop availability Possible layers: • ping server • check HTTP status ”200 OK” • check homepage title • login as user • complete order process Martin Schütte So ware Testing on the Web 2012-10-13 32 / 51
  • 33. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The next level: Behaviour Tests Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen Angenommen ich bin auf ”/” Und ich suche nach ”sponsoren” Dann ich sollte ”Premium-Sponsor Chocri” sehen Und ich sollte ”Platin-Sponsor Adobe” sehen Martin Schütte So ware Testing on the Web 2012-10-13 33 / 51
  • 34. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Next Level: Behaviour Driven Development TDD with other syntax? Is BDD the same as TDD? Yes. If you’re a programmer, and your entire team is programmers, and all your stakeholders are programmers. – Dan North We don’t call them “acceptance tests” because you can’t ask a business person “Please help me with my acceptance test”. Try “I’d like to talk to you about the scenario where…” instead. Or, “Can you give me an example?” Either of these are good. – Liz Keogh Martin Schütte So ware Testing on the Web 2012-10-13 34 / 51
  • 35. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unit & Behaviour Testing Unit Tests Behaviour Tests • unit testing • acceptance test scenarios • programmers • non-developers • programming language • language of business domain • bottom-up • top-down / outside-in • assertXYZ • X should do Y • tests derived from user stories • execute user stories ⇒ development tool ⇒ communication tool Martin Schütte So ware Testing on the Web 2012-10-13 35 / 51
  • 36. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . One Notation/“Language”: Gherkin Feature: [one line describing the story] In order to [benefit] As a [role] I want [feature] Scenario: [A scenario specific goal] Given [Something that needs to have happened or be true] When [Some task I must do] And [Another task] Then [Some way I know I’ve achieved my goal] Martin Schütte So ware Testing on the Web 2012-10-13 36 / 51
  • 37. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I18N # language: de Funktionalität: [one line describing the story] Um zu erreichen [benefit] Als [role] Möchte ich [feature] Szenario: [Title] Angenommen [context] Wenn [event] Dann [outcome] Und [another outcome] Martin Schütte So ware Testing on the Web 2012-10-13 37 / 51
  • 38. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TDD with other syntax? Scenario: New lists are empty Given a new list Then the list should be empty. Scenario: Lists with things in them are not empty. Given a new list When we add an object Then the list should not be empty. . Martin Schütte So ware Testing on the Web 2012-10-13 38 / 51
  • 39. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implementation / Translation A look behind the curtain: • framework is clever but not magical • some translation needed • statements have to become executable code Mechanism: • plain sentence → method name • quoted words → arguments • matching with annotated RegExp • methods yield success, exception, or pending exception Martin Schütte So ware Testing on the Web 2012-10-13 39 / 51
  • 40. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat/Mink FeatureContext.php use BehatMinkExtensionContextMinkContext; class FeatureContext extends BehatContext { public function __construct(array $parameters) { $this->useContext(’mink’, new MinkContext()); } } MinkContext adds web-related methods and their I18N translations, e. g.: • visit($page) • fillField($field, $value) • pressButton($button) Martin Schütte So ware Testing on the Web 2012-10-13 40 / 51
  • 41. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Output [mschuett@dagny] ~/wpcamp/code% ./vendor/bin/behat Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen # features/wpcamp.feature:8 Angenommen ich bin auf ”/” # BehatMinkExtensionContextMinkContext::visit Und ich suche nach ”sponsoren” Dann ich sollte ”Premium-Sponsor Chocri” sehen # BehatMinkExtensionContextMinkContext::asser Und ich sollte ”Platin-Sponsor Adobe” sehen # BehatMinkExtensionContextMinkContext::asser 1 scenario (1 undefined) 4 steps (1 passed, 2 skipped, 1 undefined) 0m1.574s You can implement step definitions for undefined steps with these snippets: /** * @Given /^ich suche nach ”([^”]*)”$/ */ public function ichSucheNach($arg1) { throw new PendingException(); } Martin Schütte So ware Testing on the Web 2012-10-13 41 / 51
  • 42. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Edit FeatureContext.php add method /** * @Given /^ich suche nach ”([^”]*)”$/ */ public function ichSucheNach($arg) { return array( new When(”I fill in ”s” with ”$arg””), new When(”I press ”searchsubmit””), ); } Martin Schütte So ware Testing on the Web 2012-10-13 42 / 51
  • 43. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Output [mschuett@dagny] ~/wpcamp/code% ./vendor/bin/behat Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen # features/wpcamp.feature:8 Angenommen ich bin auf ”/” # BehatMinkExtensionContextMinkContext::visit Und ich suche nach ”sponsoren” # FeatureContext::ichSucheNach() Dann ich sollte ”Premium-Sponsor Chocri” sehen # BehatMinkExtensionContextMinkContext::asser Und ich sollte ”Platin-Sponsor Adobe” sehen # BehatMinkExtensionContextMinkContext::asser 1 scenario (1 passed) 4 steps (4 passed) 0m2.59s Martin Schütte So ware Testing on the Web 2012-10-13 43 / 51
  • 44. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture (PHP) Behat PHPUnit . Mink Selenium Sahi Goutte Zombie.js Firefox Opera Martin Schütte So ware Testing on the Web 2012-10-13 44 / 51
  • 45. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cucumber original Ruby version, nicer syntax # Example Webrat step definitions When /I search for ”(.*)”/ do |query| @browser.text_field(:name, ’s’).set(query) @browser.button(:name, ’searchsubmit’).click end Then /I should see/ do |text| @browser.text.should =~ /#{text}/m end Martin Schütte So ware Testing on the Web 2012-10-13 45 / 51
  • 46. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture (Ruby) . Cucumber Selenium Watir Webrat Celerity Firefox Opera Martin Schütte So ware Testing on the Web 2012-10-13 46 / 51
  • 47. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception unit test a new full-stack testing PHP framework $I = new CodeGuy($scenario); $I->wantTo(’perform actions and see results’); $I->testMethod(’DateTime.getTimestamp’); $date = new DateTime(”2012-10-13T14:45:00+02:00”); $I->executeTestedMethodOn($date); $I->seeResultIs(’int’); $I->seeResultEquals(1350132300); Martin Schütte So ware Testing on the Web 2012-10-13 47 / 51
  • 48. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception acceptance test $I = new WebGuy($scenario); $I->wantTo(’find infos on the website’); $I->see(’Das WordPress Event’, ’html head title’); $I->fillField(’s’, ’sponsoren’); $I->click(’searchsubmit’); $I->see(’Premium-Sponsor Chocri’, ’#content’); $I->see(’Platin-Sponsor Adobe’, ’#content’); Martin Schütte So ware Testing on the Web 2012-10-13 48 / 51
  • 49. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception Output [mschuett@dagny] ~/wpcamp/code% php codecept.phar run --steps Codeception PHP Testing Framework v1.1.4 Powered by PHPUnit 3.6.10 by Sebastian Bergmann. Suite unit started Trying to perform actions and see results (exampleCept.php) Scenario: * I test method ”DateTime.getTimestamp” * I execute tested method on ”DateTime” * I see result is ”int” * I see result equals ”1350132300” OK Suite functional started Suite acceptance started Trying to find infos on the website (wpcampCept.php) Scenario: * I am on page ”/” * I see ”Das WordPress Event”,”html head title” * I fill field ”s”,”sponsoren” * I click ”searchsubmit” * I see ”Premium-Sponsor Chocri”,”#content” * I see ”Platin-Sponsor Adobe”,”#content” OK Time: 4 seconds, Memory: 9.50Mb OK (2 tests, 5 assertions) Martin Schütte So ware Testing on the Web 2012-10-13 49 / 51
  • 50. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems • most Web tests require known DB state • behaviour testing is slow • only run locally • use in-memory DB • draw the line • features in documentation (what) → add behaviour test • mere implementation (how) → only unit tests Martin Schütte So ware Testing on the Web 2012-10-13 50 / 51
  • 51. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary Value individuals and interactions over processes and tools – The Agile Manifesto Links: • PHPUnit • Sebastian Bergmann (PHPUnit) • Selenium • Dan North’s blog (BDD) • Behat & Mink • Liz Keogh’s blog (BDD) • Cucumber • Geek and Poke • Codeception • Not Invented Here Martin Schütte So ware Testing on the Web 2012-10-13 51 / 51