1. Patterns and practices of building
Gherkin based
living documentations
Building SpecLog with SpecFlow
CHRISTIAN HASSA & GASPAR NAGY
ch@techtalk.at & gn@techtalk.at
19th November 2010
2. Agenda
Walkthrough: Using SpecFlow in a project
• Gathering requirements & formulating acceptance
criteria
• Development & testing concept
• Build & release process, reports
Special challenges
• User Stories vs. Features
• Synchronization Scenarios
• Tool Widgets
• Automation Helpers
3. About Us
Project Sponsor SpecFlow
Product Owner, Coach
Twitter: @chr99ha
Gáspár Nagy
TechTalk
Committer to SpecFlow
http://gasparnagy.blogspot.com
Twitter: @gasparnagy
Christian Hassa
TechTalk
TechTalk is a software development and
consulting company with ~60 people
located in Vienna, Budapest and Zürich.
We focus on Scrum and .NET.
The research is partially supported by the
project of Eötvös Loránd University:
TÁMOP-4.2.1.B-09/1/KMR-2010-0003
6. Our Domain: SpecLog
A product developed by TechTalk
managing flow of requirements in agile projects
integrates with SpecFlow and TFS
Team size: ~ 7 people
WPF SmartClient
• central repository
or stand-alone
• occasionally
connected
7. Deriving scope from goals … is not easy
Deriving user stories from …
… vision/business goals: too far away
… actor goals: not focused enough
Roadmap helped to focus
First milestone goal:
„use SpecLog for SpecLog“
(even if it hurts …)
vision
Product
Backlog
Epic
User Story
Epic
User Story
User Story
User Story
User Story
business
goals
Roadmap
derive scope from goals
actor
goals
10. Illustrating user stories with examples:
acceptance criteria
Product
Backlog
Epic
User Story
Epic
User Story
User Story
User Story
User Story
Sprint
Backlog
illustrate
using
examples
UI Scribbles as key examples
Acceptance Criteria
as key examples
US
US
US
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
11. Refining: Formalization in Gherkin
• Formalization in pairs
• Ideal: Developer + Product Owner
• Scenario tags: @specifiedSprint
Backlog
US
US
US
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria Given
When
Then
…
…
…
refine
the
specification
UI Scribbles as key examples
specification with examples
Acceptance Criteria
as key examples
12. Automating Gherkin scenarios
Automation of Gherkin scenarios
… too costly on UI (custom WPF UI)
… instead on controller
Sprint
Backlog
US
US
US
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria
Acceptance
Criteria Given
When
Then
…
…
…
SpecFlow
{ … }
{ … }
{ … }
automate
literally
specification with examples
executable
specification
Scenario tags:
@automated
@manual
13. Development concept
In development we have to find the right balance between…
• High quality – the application will be used by a broad
audience
• Changing directions – the implementation also serves as a
test for the concept
Our goals are
• Develop the application outside-in
• Preferable AC by AC
• Automate AC through controller (view model) layer
• Design concept but no detailed design upfront
• Many things from XP (TDD, Pair programming, common
code ownership)
16. Build & Release Process, Reports
We wanted to build a confortable infrastructure that
supports our process
• CI
• Short release cycle through automated build
• Build quality measures
• Reports
17. Automated Builds - CI
Runs on every check-in (no gated check-in)
No versioning
Builds the application & runs unit tests (*.Tests)
• Fails if any of these fails
18. Automated Builds – Acceptance
Triggered by check-ins, cumulated (1h)
Verifies build quality
• Executes automated acceptance tests
• Detects version compatibility issues
Produces ready-to-use release
• Increments version number
• Produces an installer
• Generates test execution report (“living
documentation”)
• Publishes the results on a website
22. Executing Acceptance Test
Implementation of an AC can be a longer process
• Might need several days
• Might be developed by more then one person
• Execution of the entire test suite might be long
We cannot expect that all acceptance test passes all the
time
What we rather need is
• Ensure that “done” stories are still working
• Get a feedback about the stories currently
implemented
24. Testing
Developer: Smoke testing of UI
• tag user story with „TestIt“
Tester: exploratory testing of all „TestIt“ user stories
• time boxed, once a week
Limited number of manually executed workflow tests
• extended with new features
Bugs: formulized to Gherkin acceptance criteria
• when automation possible
• grouped to user story on backlog when not fixed
immediately
27. Evolving the living documentation
Product/Sprint Backlog
User Story 1
AccCrit 1
AccCrit 2
User Story 2
AccCrit 3
AccCrit 4
Living Documentation
Feature 1
AccCrit 1
AccCrit 2
Feature n
AccCrit 4
AccCrit m
User Story n
AccCrit 5
AccCrit m
AccCrit 3
AccCrit 5
„Done“
• Next possible increments of
product (units of work)
• Organized /refined according
to priority, value, effort, risk,
actor-goals, …
• Product Documentation
• Organized/refined according to
functionality and overview
• Versioned/maintained with
source code
28. Synchronization Scenarios
The application should support collaborative editing with
multiple occasionally connected clients
• This is a general cross-cutting concern
• Hard to test manually
• Essential feature of the application
Solution: we concretize this requirement for each story
where it can be applied
• This is part of a check list – so we don’t forget it
29. Synchronization Scenarios
Synchronization of a user story “value text”
@automated @done
Scenario: should synchronize value text of user story
Given a workspace for a shared repository
And the workspace is active on 'Client1'
And the repository is open on 'Client2'
And a user story is in the repository
When I set the value text of the user story to 'some value' on 'Client1'
And all changes are synchronized
Then the user story in the repository on 'Client2' should have the value
text 'some value'
30. Synchronization Scenarios
Specifying goal text in “local mode” – no need to mention clients
@automated @done
Scenario: should be able to specify value text for user story through
card
Given an active workspace for a repository
And a user story in the repository
And a card for the user story on the workspace
When I set the value text of the user story to 'some value'
Then the user story in the repository should have the value text 'some
value'
31. Synchronization Scenarios
Specifying AC headline (title) – no need to mention workspaces even
@automated @done
Scenario: Acceptance criteria can be described with a headline
Given a user story
And an acceptance criterion for the user story
When I set the headline of the acceptance criterion to 'some headline'
Then the acceptance criterion should have the headline 'some headline' in
the repository
[Given("a user story")]
public void GivenAUserStory()
{
EnsureOpenStandaloneRepository();
GivenAUserStoryIsInTheRepository();
}
32. Synchronization Scenarios
Conflict resolution – for the winner
@automated @done @conflict_resolution
Scenario Outline: first synchronized concurrent modification of requirement goal text should win
Given a workspace for a shared repository
And the workspace is active on 'Client1'
And the workspace is active on 'Client2'
And a <requirement> is in the repository
When I set the goal text of the <requirement> to 'some goal1' on 'Client1'
And I set the goal text of the <requirement> to 'some goal2' on 'Client2'
And all changes are synchronized by 'Client1'
And all changes are synchronized by 'Client2'
And all changes are synchronized by 'Client1'
Then the <requirement> in the repository should have the goal text 'some goal1'
Examples:
| requirement |
| business goal |
| actor-goal |
| user story |
33. Synchronization Scenarios
Conflict resolution – for the looser
@automated @specified @conflict_resolution
Scenario Outline: concurrent modifications of goal text of requirements that lose should be
captured as conflicts
Given a workspace for a shared repository
And the workspace is active on 'Client1'
And the workspace is active on 'Client2'
And a <requirement> is in the repository
And the <requirement> has no conflicts
When I set the goal text of the <requirement> to 'some goal1' on 'Client1'
And I set the goal text of the <requirement> to 'some goal2' on 'Client2'
And all changes are synchronized by 'Client1'
And all changes are synchronized by 'Client2'
Then the <requirement> in the repository should have the conflict 'conflicting goal: some
goal2' introduced by 'Client2'
Examples:
| requirement |
| business goal |
| actor-goal |
| user story |
34. Dealing with cross-cutting concepts
Tool-widget
• Floats over workspace
• Can be open/closed using control bar
• Position kept per user session
35. Formalizing tool widget concept in Gherkin
examples for
tool widget
concept
examples of
tool widgets
(extended over time)
manual test
that there is a
specific tool widget
instance
36. TestContext, TestClient, TestServer &
Automation Drivers
When automating acceptance criteria, similar tasks have
to be solved from different step bindings
• These should also communicate with each other
Given a user story
And an acceptance criterion for the user story
• TestContext is a class that is injected into the step
binding classes to provide a platform for the
communication
• TestClient, TestServer – entry points for the
automation of a client or a server instance
• Automation Drivers – grouping together the
automation code for automating a class (view model)
