How to support acceptance testing of web applications with the PHP tool Behat in a BDD manner. Including an overview of acceptance testing in agile software development in general. Concluding with the most frequent challenges of using Behat and similar tools in practice.

1. AUTOMATED
WEB ACCEPTANCE TESTING
WITH BEHAT
Fabian Kiss
YMC AG | May 2012
Picture by viZZZual.com, licensed under the CC BY 2.0

2. IN AGILE
TESTING
Picture by Sheffield Tiger, licensed under the CC BY 2.0

3. MIKE COHN'S AGILE TESTING PYRAMID
Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
at ed
om
ut s
A t
t es

4. MIKE COHN'S AGILE TESTING PYRAMID
Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
UI
Service
Unit

5. MIKE COHN'S AGILE TESTING PYRAMID
Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
least investment
UI
Service
most investment
Unit

6. MIKE COHN'S AGILE TESTING PYRAMID
Succeeding with Agile: Software Development Using Scrum, Addison-Wesley
least investment
UI
Service
most investment
Unit

7. BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
tests
ated
ut om
al +a
nu
Ma

8. BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
Acceptance Usability
Testing Testing
... ...
Performance
Unit Testing
Testing
...
…

9. BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
Business-facing
Acceptance Usability
Support programming
Testing Testing
Critique product
... ...
Performance
Unit Testing
Testing
...
…
Technology-facing

10. BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
Business-facing
Acceptance Usability
Support programming
Testing Testing
Critique product
... ...
Performance
Unit Testing
Testing
...
…
Technology-facing

11. tests
ated
ut om
al +a
nu
Ma

12. tests
ated
ut om
al +a
nu
Ma

13. BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
Acceptance Usability
Testing Testing
... ...
Performance
Unit Testing
Testing
...
…

14. BRIAN MARICK'S AGILE TESTING MATRIX
http://www.exampler.com/old-blog/2003/08/21/
Acceptance Usability
BDD Testing Testing
... ...
Performance
TDD Unit Testing
Testing
/BDD ...
…

15. BDD

16. Specification-oriented BDD
BDD
Scenario-oriented BDD

17. Specification-oriented BDD RSpec
BDD
Scenario-oriented BDD

18. Specification-oriented BDD RSpec
BDD
Cucumber
Scenario-oriented BDD
Behat

19. Specification-oriented BDD RSpec
BDD
Cucumber Gherkin DSL
Scenario-oriented BDD
Behat

20. Specification-oriented BDD RSpec
BDD
Cucumber Gherkin DSL
Scenario-oriented BDD
Behat

21. Specification-oriented BDD RSpec
...
BDD
Cucumber Gherkin DSL
Scenario-oriented BDD
Behat
...

22. GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
Then some testable outcome is achieved
And something else we can check happens too

23. GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
Then some testable outcome is achieved
And something else we can check happens too

24. GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
Then some testable outcome is achieved
And something else we can check happens too

25. GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
Then some testable outcome is achieved
And something else we can check happens too

26. GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
Then some testable outcome is achieved
And something else we can check happens too

27. GHERKIN SYNTAX + SEMANTICS
http://docs.behat.org/guides/1.gherkin.html
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
Then some testable outcome is achieved
And something else we can check happens too

28. GHERKIN EXAMPLE
http://docs.behat.org/guides/1.gherkin.html
Scenario: Wilson posts to his own blog
Given I am logged in as Wilson
When I try to post to "Expensive Therapy"
Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog
Given I am logged in as Wilson
When I try to post to "Greg's anti­tax rants"
Then I should see "Hey! That's not your blog!"

29. GHERKIN EXAMPLE
http://docs.behat.org/guides/1.gherkin.html
Feature
Scenario: Wilson posts to his own blog
Given I am logged in as Wilson
When I try to post to "Expensive Therapy"
Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog
Given I am logged in as Wilson
When I try to post to "Greg's anti­tax rants"
Then I should see "Hey! That's not your blog!"

30. STEPS & STEP DEFINITIONS
http://docs.behat.org/quick_intro.html#basic-usage
Given some precondition
/**
* @Given /^some precondition$/
*/
public function somePrecondition()
{
//...
}

31. STEPS & STEP DEFINITIONS
http://docs.behat.org/quick_intro.html#basic-usage
Given some precondition
Step
Step
/**
definition
* @Given /^some precondition$/
*/
public function somePrecondition()
{
//...
}

32. STEPS & STEP DEFINITIONS
http://docs.behat.org/quick_intro.html#basic-usage
Given some precondition
Step
Matching
Step
/**
definition
* @Given /^some precondition$/
*/
public function somePrecondition()
{
//...
}
Run

33. STEP ARGUMENTS
http://docs.behat.org/quick_intro.html#some-more-behat-basics
When I try to post to "Expensive Therapy"
/**
* @When /^I try to post to "([^"]*)"$/
*/
public function iTryToPostTo($site)
{
//...
}

34. STEP ARGUMENTS
http://docs.behat.org/quick_intro.html#some-more-behat-basics
When I try to post to "Expensive Therapy"
Argument
/**
* @When /^I try to post to "([^"]*)"$/
*/
public function iTryToPostTo($blog)
{
//...
}

35. SCENARIO BACKGROUNDS
http://docs.behat.org/guides/1.gherkin.html#backgrounds
Background:
Given I am logged in as Wilson
Scenario: Wilson posts to his own blog
When I try to post to "Expensive Therapy"
Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog
When I try to post to "Greg's anti­tax rants"
Then I should see "Hey! That's not your blog!"

36. SCENARIO BACKGROUNDS
http://docs.behat.org/guides/1.gherkin.html#backgrounds
Background:
Given I am logged in as Wilson
Scenario: Wilson posts to his own blog
When I try to post to "Expensive Therapy"
Then I should see "Your article was published."
Scenario: Wilson fails to post to somebody else's blog
When I try to post to "Greg's anti­tax rants"
Then I should see "Hey! That's not your blog!"

37. SCENARIO OUTLINES
http://docs.behat.org/guides/1.gherkin.html#scenario-outlines
Scenario Outline: Wilson posts to a blog
Given I am logged in as Wilson
When I try to post to "<blog>"
Then I should see "<authorization>"
Examples:
| blog | authorization |
| Expensive Therapy | Your article was published. |
| Greg's anti­tax rants | Hey! That's not your blog! |

38. SCENARIO OUTLINES
http://docs.behat.org/guides/1.gherkin.html#scenario-outlines
Scenario Outline: Wilson posts to a blog
Given I am logged in as Wilson
When I try to post to "<blog>" x2
Then I should see "<authorization>"
Examples:
| blog | authorization |
| Expensive Therapy | Your article was published. |
| Greg's anti­tax rants | Hey! That's not your blog! |

39. STEP TABLES
http://docs.behat.org/guides/1.gherkin.html#tables
Scenario: Wilson posts to a blog
Given I am logged in as Wilson
When I try to post to:
| blog |
| Expensive Therapy |
| Greg's anti­tax rants |
Then I should see an authorization outcome

40. STEP TABLES
http://docs.behat.org/guides/1.gherkin.html#tables
Scenario: Wilson posts to a blog
Given I am logged in as Wilson
When I try to post to:
| blog | x2
| Expensive Therapy |
| Greg's anti­tax rants |
Then I should see an authorization outcome

41. ACCESSING THE WEB
Picture by Misserion, licensed under the CC BY 2.0

42. MINK API
http://mink.behat.org/
Driver object
$client = new SeleniumClient($host, $port);
$driver = new BehatMinkDriverSeleniumDriver(
'firefox', 'base_url', $client
);
Session object
$session = new BehatMinkSession($driver);
$session­>start();
$session­>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session­>getPage();
$elementByCss = $page­>find('css', 'h3 > a');
$elementByCss­>click();

43. MINK API
http://mink.behat.org/
Driver object
$client = new SeleniumClient($host, $port);
$driver = new BehatMinkDriverSeleniumDriver(
'firefox', 'base_url', $client
);
Session object
$session = new BehatMinkSession($driver);
$session­>start();
$session­>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session­>getPage();
$elementByCss = $page­>find('css', 'h3 > a');
$elementByCss­>click();

44. MINK API
http://mink.behat.org/
Driver object
$client = new SeleniumClient($host, $port);
$driver = new BehatMinkDriverSeleniumDriver(
'firefox', 'base_url', $client
); Sahi, Goutte, Zombie.js
Session object
$session = new BehatMinkSession($driver);
$session­>start();
$session­>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session­>getPage();
$elementByCss = $page­>find('css', 'h3 > a');
$elementByCss­>click();

45. MINK API
http://mink.behat.org/
Driver object
$client = new SeleniumClient($host, $port);
$driver = new BehatMinkDriverSeleniumDriver(
'firefox', 'base_url', $client
); Sahi, Goutte, Zombie.js
Session object
$session = new BehatMinkSession($driver);
$session­>start();
$session­>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session­>getPage();
$elementByCss = $page­>find('css', 'h3 > a');
$elementByCss­>click();

46. MINK API
http://mink.behat.org/
Driver object
$client = new SeleniumClient($host, $port);
$driver = new BehatMinkDriverSeleniumDriver(
'firefox', 'base_url', $client
); Sahi, Goutte, Zombie.js
Session object
$session = new BehatMinkSession($driver);
$session­>start();
$session­>visit('http://my_project.dev/some_page.php')
Exemplary interaction
$page = $session­>getPage();
$elementByCss = $page­>find('css', 'h3 > a');
$elementByCss­>click();

47. BEHAT REPORT
~/Behat$ behat features/blogpost.feature
Feature: Blog posting
As a blogger
I want to post an article to a blog
Scenario: Wilson posts to his own blog # features/blogpost.feature:5
Given I am logged in as "Wilson" # FeatureContext::iAmLoggedInAs()
When I try to post to "Expensive Therapy" # FeatureContext::iTryToPostTo()
Then I should see "Your article was published." # FeatureContext::iShouldSee()
Scenario: Wilson fails to post to somebody else's blog # features/blogpost.feature:10
Given I am logged in as "Wilson" # FeatureContext::iAmLoggedInAs()
When I try to post to "Greg's anti­tax rants" # FeatureContext::iTryToPostTo()
Blog "Greg's anti­tax rants" does not exist
Then I should see "Hey! That's not your blog!" # FeatureContext::iShouldSee()
2 scenarios (1 passed, 1 failed)
6 steps (4 passed, 1 skipped, 1 failed)

48. BEHAT REPORT
~/Behat$ behat features/blogpost.feature ­­format pretty
Feature: Blog posting
As a blogger
I want to post an article to a blog
Scenario: Wilson posts to his own blog # features/blogpost.feature:5
Given I am logged in as "Wilson" # FeatureContext::iAmLoggedInAs()
When I try to post to "Expensive Therapy" # FeatureContext::iTryToPostTo()
Then I should see "Your article was published." # FeatureContext::iShouldSee()
Scenario: Wilson fails to post to somebody else's blog # features/blogpost.feature:10
Given I am logged in as "Wilson" # FeatureContext::iAmLoggedInAs()
When I try to post to "Greg's anti­tax rants" # FeatureContext::iTryToPostTo()
Blog "Greg's anti­tax rants" does not exist
Then I should see "Hey! That's not your blog!" # FeatureContext::iShouldSee()
2 scenarios (1 passed, 1 failed)
6 steps (4 passed, 1 skipped, 1 failed)
default formatter (“pretty”)
all formatters:
http://docs.behat.org/guides/6.cli.html#format-options

49. CHALLENGES...

50. Picture by Marjan Krebelj, licensed under the CC BY 2.0
DEVELOPERS WRITE
SCENARIOS

51. DEVELOPERS WRITE
SCENARIOS
Picture by Marjan Krebelj, licensed under the CC BY 2.0
AN ARTIFACT ASKED FOR?
UNDERSTANDABILITY?

52. SCENARIO ABSTRACTION LEVEL
Picture by Conor Lawless, licensed under the CC BY 2.0

53. Picture by Conor Lawless, licensed under the CC BY 2.0
SCENARIO ABSTRACTION LEVEL t name”
t name into field “firs
fill my firs name”
When field “last
VS.
I Wh en I name into
the f fill ill my last
or m When I f
..
When I fill .

54. REDUNDANCY
Picture by Rob Faulkner, licensed under the CC BY 2.0

55. REDUNDANCY
Picture by Rob Faulkner, licensed under the CC BY 2.0
LIMITED: GHERKIN'S REUSE CAPABILITIES
LIMITED: UNDERSTANDING OF THOSE

56. Picture by Earls37a, licensed under the CC BY 2.0
TIME-CONSUMING REFACTORING

57. Picture by Earls37a, licensed under the CC BY 2.0
EXECUTION IN REAL-TIME
TIME-CONSUMING REFACTORING

58. COMPLEX INFRASTRUCTURE
Picture by Domenico Nardone, licensed under the CC BY 2.0

59. Picture by Domenico Nardone, licensed under the CC BY 2.0
COMPLEX INFRASTRUCTURE
Behat Mink Driver Browser Emulator Browser

60. FURTHER REFERENCES:
 http://www.softwarequalityconnection.com/2011/08/how-to-improve-communication-between-qa-and-development/
 http://dannorth.net/introducing-bdd/
 http://www.infoq.com/news/2011/02/BDD-ATDD
 http://www.slideshare.net/headrevision/the-concept-of-behaviordriven-development
 http://www.slideshare.net/everzet/bdd-in-symfony2
SOME BEST PRACTICES:
 http://www.agileengineeringdesign.com/2012/01/7-deadly-sins-of-automated-software-testing/
 http://elabs.se/blog/15-you-re-cuking-it-wrong
 http://www.engineyard.com/blog/2009/15-expert-tips-for-using-cucumber/
 http://eggsonbread.com/2010/09/06/my-cucumber-best-practices-and-tips/