2. 3
3
ABOUT THE SPEAKER
Josh has over seven years of experience as an
accomplished software testing and development
professional.
Josh Eastman
Director, Strategic Solutions
JEastman@kforce.com
4. 5
GROUP EXERCISE
• Imagine you are a QA manager
• You have 25 direct reports (QA Analysts)
• 6 Agile lines – 2 Run (Production Support) + 4 Build
• Business demand is expected to nearly double in the next year
• System defect rate = 5 defects / 1,000 release hours
• Production defect rate = 1.75 defects / 1,000 release hours
How can you possibly prepare for the increase in business demand
and try to lower your defect rates – or increase quality –
simultaneously?
5. 6
SOLUTION
Business Demand Increases By 187% - 354,739 release hours compared to 190,008
release hours
System Defect Rate Decreased 1.43 defects/1000 release hours compared to 5
defects/1000 release hours
Production Defect Rate Decreased 1.3 defects/1000 release hours compared to 1.75
defects/1000 release hours
Direct Reports Increased Only by 7 – 32 compared to 25
Fortune 100 Company Year X Year X+1
Business Demand (Release Hours) 190,008 354,739
System Testing Defects 937 508
Production Defects 332 464
Testing Resources 25 32
Annual Savings
$1,191,360
$1,183,540
$1,365,900
TOTAL ANNUAL
BUSINESS VALUE ($)
$3,740,800
Employ a deliberate emphasis on quality – implementing
an industry best practice – Behavior Driven Development (BDD) –
with a determined focus on incorporating automation.
7. 8
MOST ORGANIZATIONS USE
SOME FORM OF AGILE
Agile emphasizes empirical feedback, team self management, and
striving to build properly tested product increments within short
iterations.
8. 9
DIFFERENCES BETWEEN TDD,
BDD + ATDD
BDD
TDD
ATDD
Tests focused on what a Class or Method execution
results. What result should it return?
Tests focused on behavior of classes or
components. What behavior should it exhibit?
Code
Requirements
Internal External
Tests focused on requirements of applications.
What business functionality should it implement?
A story’s behavior is simply its acceptance criteria – if the system fulfills
all the acceptance criteria, it’s behaving correctly; if it doesn’t, it isn’t.
Acceptance Test Driven Development (ATDD) is a form of Behavior
Driven Development (BDD) that focuses on requirements specified in the
form of acceptance criteria. How are Testing and Requirements related?
9. 10
WHAT IS BDD?
BDD promotes
requirements by example,
collaboration, lower cost in
resolving defects, and
automation in the form of
business value.
• Industry practice in which whole
team collaborates on system
testing and “definition of done”
• Developer focuses on making
system test(s) pass
• Automate tests while production
code is being developed
• Automation design completes
before or parallel to development
• Tests become part of build pipeline
and are run throughout the sprint
• Automated tests represent the true
business value and living
documentation
10. 11
AUTOMATION STRATEGY OLD
• Often a one-tool approach
• Brittle automated tests
• Maintenance is high
• Defects escape easily
• Developer accountability low
• Cost of quality is high
More
Automated
Less
Automated
• Tools have not been plentiful
• Experience with open source low
These problems have caused automation to be expensive and
ineffective.
11. 12
AUTOMATION STRATEGY NEW
• Use tools such as xUnit
• Find defects quickly
• Keep technical debt low
• Use tools such as Selenium, Watir
• Make UI tests maintainable
• Can increase UI tests with stability
• Use tools such as Cucumber
• Best use in regression tests
• Use plug-ins to extend to other tiers
More
Automated
Less
Automated
Unit Testing provides the most effective way in preventing defects –
increasing and maintaining quality.
13. 14
HOW DOES BDD INCREASE
QUALITY?
1. Process Encourages Collaboration
2. Requirements with Examples
3. Promotes Automation
4. Find Defects Earlier and Often
There are many other values to BDD but we’ll discuss these four values
specifically.
How Does BDD Increase Quality?
14. 15
BDD VALUE #1
PROCESS ENCOURAGES COLLABORATION
• BDD requires discipline to ensure that the right people work together
to create the tests
• Customers must work with developers
to prevent the developers from writing tests that are too low level
and tests use customer terminology is used
to ensure that what is being tested is important (to the customer)
• Developers must work with testers
testers will be looking for corner cases and good coverage
developers can use their expertise to ensure
that the test cases are properly decomposed
• BA, Dev, and QA roles will balance each
other
Collaboration is primary
Consistent domain vocabulary is critical
Getting automation right is important
15. 16
BDD VALUE #2
REQUIREMENTS WITH EXAMPLES
• Uses concrete and specific examples
written in Gherkin which promotes:
Conversation between the business,
analyst, developer, and tester
(3 Amigos)
Prompts early questions around whether
the scenarios are accurate
Captures the conversation, with the
benefit of providing testable functional
tests
• Focuses on thin slices of system behavior
that deliver business value which are
potentially automatable without translation
• Develops executable specifications
captured in live documentation that can
serves as requirements, manual tests,
automation scripts
16. 17
WHAT IS GHERKIN?
Gherkin is a language/syntax that is a Business Readable,
Domain Specific Language created specifically for behavior
descriptions.
• It gives you the ability to remove logical details from behavior.
Gherkin serves as your project's documentation as well as your project's
automated tests.
• Test template in the form of Given When Then (GWT)
• Given some initial context (the givens)
• When an event occurs
• Then ensure some outcomes
Given a user navigates to the Kforce website
And the user fills in “automation engineer” for job title or skill
And the user fills in “43017” for city state zip
When the user clicks the search button
And the application navigates to the Search Jobs page
Then the page displays the save as a job alert button element
Then the page displays the reset search button element
Real World Example
17. 18
WHAT IS CUCUMBER?
Cucumber is an open source software tool written in
Ruby and uses a plain text Domain Specific Language
(DSL) called Gherkin.
• Cucumber lines of text are processed by Ruby ‘step definitions’ which are code
blocks that execute the given, when, and then specifications against an
application. It runs automated acceptance tests written in a behavior driven
development (BDD) / acceptance test driven development (ATDD) style.
• Cucumber lets software development teams describe how software should
behave in plain text. The text is written in a business-readable “plain English”
type language and serves as documentation, automated tests and
development-aid - all rolled into one format.
• Cucumber works with Ruby, Java, .NET, Flex or web applications written
in any language. It has been translated to over 40 spoken languages.
Aslak Hellesoy created Cucumber to support BDD/ATDD.
He states: The idea was to combine automated tests, functional
requirements and software documentation into one format that would be
understandable by non-technical people as well as testing tools.
18. 19
BDD VALUE #3
PROMOTES AUTOMATION
Requirements Traceability
Selective Testing
Execution Reports
Cross-Browser Testing Cross-Environment Testing
DEV QA STAGE
IT ST PT UAT
Continuous Integration