We've all been there. You work incredibly hard to develop a feature and design tests based on written requirements. You build a detailed test plan that aligns the tests with the software and the documented business needs. When you put the tests to the software, it all falls apart because the requirements were updated without informing everyone. But help is at hand. Enter business-driven development and Cucumber, a tool for running automated acceptance tests. Join Mary Thorn as she explores the nuances of Cucumber and shows you how to implement specification-by-example, behavior-driven development, and agile acceptance testing. By fostering collaboration for implementing active requirements via a common language and format, Cucumber bridges the communication gap between business stakeholders and implementation teams. If you experience developers not coding to requirements, testers not getting requirements updates, or customers who feel out of the loop and don't get what they ask for, be here!
2. Mary Thorn
Deutsche Bank
VP of QA at Deutsche Bank Global Technologies in North Carolina, Mary Thorn has a
broad testing background that spans automation, data warehouses, and web-based
systems in a wide variety of technologies and testing techniques. During her more than
fifteen years of experience in healthcare, HR, and SaaS-based products, Mary has held
manager and contributor level positions in software development organizations. She
has a strong interest in agile testing methodologies and direct experience leading agile
teams through Scrum adoption and beyond. Mary is a Certified ScrumMaster and an
active participant in Triangle Institute of Software Quality Assurance.
.
3. Mary Thorn
QA Manager at Deutsche Bank.
Over 15 years of QA experience working in the following
areas: Healthcare HR Agriculture Financial and SaaS
Healthcare, HR, Agriculture,
SaaSbased products.
Have a broad testing background that spans
automation, data warehouses to web-based systems in
a wide variety of technologies and testing techniques.
Heavy interest in Agile Testing Methodologies and direct
experience leading Agile teams in the SCRUM Software
Development Lifecycle (SDLC).
1
4. •
Introduction
•
The Communication Problem
•
What are Acceptance Tests?
•
Simple Example
•
Acceptance Testing with Cucumber
•
Feature Files
•
Wrap up
Goal: Presenter would like to have a successful
presentation
Acceptance Criteria:
1. Verify the attendees stay awake
2. Verify the attendees are happy with the
presentation
2
5. The Challenge:
Today, the most difficult problem in software
development is knowing what to build, not how
to build it.
Knowing what to build, requires knowing why it
is being built. In other words, understanding the
goal.
The Problem:
Stories typically concentrate on the what and
the how, not the why. Goals and examples are
usually missing.
Stories are usually a list of imperative
requirements (acceptance criteria).
Imperative requirements are usually interpreted
in subtly different ways by each person. If goals
and examples are absent, misinterpretation is
much more likely.
3
6. Example:
Requirement: Draw a star with 10 points.
OR
Solution: Specification by Example
People understand requirements best using
concrete examples.
Examples clear up ambiguity and
misunderstandings.
Examples expose missing requirements.
Examples force everyone to think harder about
a problem.
4
7. Specification by Example
This is the basis of acceptance testing and
Behavior Driven Design (BDD).
It is critical that everyone participates in creating
the examples so that the understanding is
shared.
Ultimately, acceptance testing is less about the
example itself and more about the conversation
required to create the example.
What does this have
to do with automated
testing?
5
8. In a nutshell, automated
acceptance tests are
t
t t
executable examples that
define the system's
specifications
specifications.
Acceptance Tests are
Formal examples that define requirements
requirements.
Executable.
Self-explanatory and readable by all team
members.
Tests of business rules
rules.
A living specification of system behavior.
6
9. Acceptance Tests do NOT
Test pieces of code in isolation They are not a
isolation.
substitute for unit and integration tests.
Verify the entire system.
Verify all permutations of a piece of functionality.
Use mock objects except to remove a
dependency on an external 3rd party.
Benefits:
Higher quality b/c everyone shares the same
understanding of the requirements and builds it right the
first time.
Tests become the system's regression suite.
Create an objective verification of “done-ness” for a
story. A story is done when all tests pass.
Create transparency into p g
p
y
progress on a story.
y
Manual testers are part of the automation process.
Allows for more exploratory testing b/c happy path is
automated
7
10. Acceptance Criteria: Transactions are rounded
to the nearest cent.
Original Value
Rounded Value
$0.021
$0.02
$0.025
$0 025
$
$0.02
$0.029
$0.02
Why was there confusion?
The goal was missing...
Goal: Create a currency conversion system.
The Concrete Example clarified what was meant
by the acceptance criteria.
criteria
Refined Acceptance Criteria: Transactions are
always rounded down to the nearest cent.
8
11. This simple example is a REAL EXAMPLE in a
system that was well-tested and approved by all
y
pp
y
parties.
Attacker was able to steal > $15,000 starting
with a single cent using this process...
$0.01 → 0.0051 € rounded to 0.01 €
0.01 € → $0.0196 rounded to $0.02
A seemingly minor ambiguity in the
requirements was very expensive.
Concrete examples drive understanding and
consensus.
To create good representative examples,
everyone must understand the goal of the story.
For technical people to properly understand the
goal, they must understand the business
domain.
domain
9
12. What does
at
Cucumber have to
do with this?
• Cucumber lets software development teams
describe how software should behave in plain
text. The text is written in a business-readable
domain-specific language and serves as
documentation of requirements, automated
regression tests and development-aid - all
rolled into one format.
y,
,
,
• Cucumber works with Ruby, Java, .NET, Flex
or web applications written in any language. It
has been translated to over 40 spoken
languages.
10
13. Components:
•
•
•
•
•
Cucumber (Framework)
HTMLUnit (Browser Emulator)
Selinium/Watir (Browser Emulator API)
Features (Tests written in sentence format)
Step Definitions (Backend code to support
Features)
Command line –
Rake TAGS @abc
Cucumber
Framework - Tags
Terminal
Result Output-
Feature File Sentence
Browser Emulator
Step Definition File –
Matching Sentence
Selinium/Watir –
Execute Command
11
14. Process:
• Product Owner writes acceptance tests in
p
feature file scenarios format in the story. Worst
case, they at least start the Scenario:
• During grooming the team discusses the
scenarios and asks questions. Team can
add/remove scenarios during grooming or
team updates feature file after.
after
• Once Sprint has started QA and Dev work on
getting the feature file 90% complete before
the developer starts coding.
Process:
• The developers code the scenarios in the
feature file in a TDD manner.
• QA expands the scenarios in the feature files
after development if there are new findings.
• When testing is complete PO acceptance is
done by running the acceptance tests.
12
15. Two Components:
• The Feature files which contain the test
Scenarios.
• The Step Definition files which contain the Ruby
“glue” code that connects the tests to the
application.
Feature File Overview:
Written in plain text
Define the test suite and contain the actual tests
which are called Scenarios
Written collaboratively by everyone on the team
(PO, QA, and Dev)
QA can run these manually.
13
16. Feature:
Suite is defined by keyword “Feature:” followed
Feature:
by a description. This states the goal for the
Feature File. When the Feature File is
complete, a tag is included at this level so that
all of the test Scenarios contained within the file
can be run from a single command.
g
Feature: This tests the currency conversion
system
Background:
Background:
“Background:” is run before each test so that
you don’t have to repeat steps. Equal to a setup
step.
Background:
Given I login with the test account
And I navigate to the Transaction Screen
Then I should see the Transaction logo
14
17. Scenario:
Individual tests are defined by “Scenario:”
Scenario:
followed by a unique name. Scenarios are
concrete examples of how we want the software
to behave. The first line states the goal of the
given Scenario, while every line thereafter lists
the steps needed to complete the test.
p
p
Scenario: Verify Transactions are rounded
to the nearest cent.
Scenario Outline:
“Scenario Outline:” is used for data driven tests
Scenario Outline:
tests.
Scenario Outline: Verify Transactions are rounded to the nearest cent.
Given I am on the order screen
When I make a <transaction> of values
Then the <values> are rounded
Examples:
|transaction|values|
|t
ti | l
|
|.021|.02|
|.025|.02|
|.029|.02|
15
18. Keywords:
•
Tests have reserved words that begin each line:
Given, When, Then, And, But.
• Use Given to state a precondition
• Use When & And when taking an action
• Use Then when asserting a validation
Scenario Outline: Verify Transactions are rounded to the nearest cent.
Given I am on the order screen
When I make a transaction of 9.025
And save the transaction
Then the value should equal 9.02
Tags:
• Tags a powerful tool to organize tests and
scenarios. A Scenario or feature can have as
many tags as you like. Just separate them with
spaces.
• A note to the wise: Keep it simple.
@tag @smoke_test @component
16
19. @Cucumberpresentation
Feature: User would like to have a successful presentation
Tests the following acceptance criteria:
1. Verify the attendees are awake
2.
2 Verify the attendees are happy with the presentation
@Attendeeawake
Scenario: Verify the attendees are awake
Given that I am an attendee
When I have drunk caffeine
Then I will be able to listen to the presenter
@Attendeeishappy
Scenario: V if Att d
S
i Verify Attendees are happy
h
Given I am the presenter
When I talk slow
And my content is interesting
Then the attendees will be happy
Cucumber is a functional test automation tool for lean and
agile teams.
It supports behavior-driven development, specification by
p
g
p
g
example and agile acceptance testing.
You can use it to automate functional validation in a form that
is easily readable and understandable to business users,
developers and testers.
It helps teams create executable specifications, that are a
goal for development, acceptance criteria and functional
regression checks for future changes.
Allows teams to create living documentation, a single
authoritative source of information on system functionality
that is always up-to-date.
Allows manual testers to be involved in automation, as well
as, frees them up from testing the “happy path” to be able to
real exploratory testing.
17
