3. Emerging Trends in Software Testing
What we are seeing…
• Rapid delivery of software is necessitating rapid delivery and testing of code
• Change is part of the development process
• Rapidly changing technology is forcing new ways of testing
• Evolving testing tools
• Agile transformation is a cultural change
• Business pushing IT
• Emphasis on testing from a User/Business Point-of-View
Testing is often the “Achilles Heel” of agile teams because…
• Testing is often the last to be brought onto agile teams
• Testing Teams don’t have the right testing tools needed to “go agile”
• Testers often lack automation skills
• Adversarial relationship with Development Team
• Historical emphasis on up-front analysis of entire Project
• “A Testers job is to find, log, and manage defects.”
www.us.sogeti.com
Local Touch – Global Reach
3
4. Agile Testing – What is “transformed”?
Agile
Process
Time
Waterfall
Iterative
4, 12, 36 Month Projects
Strategy
Written Once
Test Plan
Per Project; Detailed; Fixed Scope
Risk
Documentation
Assessed Upfront
Test Documentation
Incremental
Parallel
Test Driven
2-4 Week Sprints
Evolving
Sprint Planning; Changing Scope
Assessed Continuously
Working tests
Defects
Logged, Managed
A Conversation
Metrics
By Project; By App
Team-based
Resources
Multiple Projects & Apps
One App; One Project at a Time
www.us.sogeti.com
Local Touch – Global Reach
4
6. The “new” requirements
User
Story
Acceptance
Criteria
Test
Cases
Gherkin
Scripts
• States what the user wants (functionality)
• Why they need it (business case)
• Boundaries of the Requirement
• The “rules”
• Definition of “Done”
• Different Variation of the functionality
• Examples of the functionality
• How to do the functionality
• Step-by-step, action-by-action
www.us.sogeti.com
Local Touch – Global Reach
6
7. The Waterfall to Agile Paradigm Shift
Waterfall
Fixed
Agile Development
Requirements
Resources
Value
Driven
Plan
Driven
Estimated
Resources
Time
The Plan creates
cost/schedule estimates
Local Touch – Global Reach
Time
Features
Release themes &
feature intent drive
estimates
www.us.sogeti.com
7
8. The Agile software tester
Task
“Traditional” tester
“New” tester
Team Role
Test-Only
Involved in all “phases”
When to test
Silo’d – scripts written and
executed at the end of the
project
ATDD – scripts written and
executed before code is
deployed
What to Test
Functional, End-to-end test,
Regression
Testing Pyramid/
4 Quadrants of Testing
Script Writing
What code is
(click-by-click)
How code behaves
(Gherkin Scripts)
Script Execution
Manual
Automated
Tools
Excel spreadsheets, manual
scripts
Automation frameworks
www.us.sogeti.com
Local Touch – Global Reach
8
9. Amigo Review Time
•
At some point during the Sprint the Team Leads (if you have Leads. If not,
bring the entire team) need to get together to discuss each User Story in
the Product Backlog
3 Amigos (typically) consist of the Analyst or Product Owner, the
Developer, and the Tester
•
Once a User Story is small enough, the
Amigo’s need to discuss:
Does the User Story contain enough
information
Are the Acceptance Criteria clear
Possible Test Cases/Scenarios
Gherkin Script(s)
Other testing that might be needed
Risk
•
Is your point estimate still accurate?
9
Local Touch – Global Reach
www.us.sogeti.com
9
10. The BDD/ATDD Timeline
• (ATDD cycle model adapted from one created by James Shore with changes
suggested by Grigori Melnick, Brian Marick, and Elisabeth Hendrickson)
www.us.sogeti.com
Local Touch – Global Reach
10
11. The Testing Pyramid
Created by Mike Cohn in Succeeding with Agile
Essential points:
1. Unit Tests form the base of your testing strategy.
2. You should have more Unit Tests than any other kind of test.
3. There is no direct relationship between Unit and GUI tests.
www.us.sogeti.com
Local Touch – Global Reach
11
12. Brian Marick’s Agile Testing Quadrants
• Goal: to ensure we accomplish all testing goals
•
Q1 - Technology-facing tests that support the team
•
Q2 - Business-facing tests that support the team
•
Q3 - Business-facing tests that critique the product
•
Q4 - Technology-facing tests that critique the product
www.us.sogeti.com
Local Touch – Global Reach
12
13. What are Gherkin scripts?
• Gherkin Scripts: connects the human concept of cause and effect to the
software concept of input/process/output.
> Given – indicates something that we accept to be true in a scenario
> When – indicates the event in a scenario
> Then – indicates an expected outcome
• Can also use “And” and “But” steps to help make a Given, When, or Then
more descriptive
• It is “business readable” – business users can understand it and follow what is
happening just as easily as Analysts, Developers, and Testers
• It a “domain specific language” – it can be used across programming
languages
www.us.sogeti.com
Local Touch – Global Reach
13
14. Examples of Gherkin Scripts:
Scenario: Successful withdrawal from an account
Given I have deposited $80 in my Account
When I request $20 from an ATM Machine
Then I should receive $20
Scenario: Draw then change a bond
And click on the canvas and release
And click on the canvas discretely to the right of the 1 node and release
And on the Bond Selection Area select the single bond
And click the 1 node and drag to the 2 node and release
When on the Bond Selection Area select the double bond
And select the 1-2 bond
Then the 1-2 bond is a double bond
Scenario: Grumpy cat pumpkins
Given I navigate to the Google Home Page
When I enter grumpy cat pumpkin
And I click the Search button
Then I will have Orange and Black pumpkins in the results
www.us.sogeti.com
Local Touch – Global Reach
14
15. Sample Automation Checklist
Must Have/
Nice to Have
1
1a
Specific Automation Functionality
How tool functions during a Sprint
Does the tool allow you to write an automated script to test this BEFORE the code is written? New functionality testing
1b
1c
Adjust scripts on the fly based on, i.e., quantity available
"Randomly" pick items (i.e. second item in search results)
1d
Handling a "change" in script during the Sprint
Must Have
1e
Capture data & re-use later in script (or other scripts)
Must Have
1f
h
Run all scripts on multiple browsers on multiple OS's:
IE, Chrome, Firefox, mobile Safari (on a mobile OS), Safari (on Apple OS X), POS
-integration with online/cloud testing sites to fill browser/OS gaps
clarification: if we need to do specific testing on a new type of phone, does the tool have the ability to test this another
way?
Test an existing defect
1i
1j
1k
-on multiple browsers & OS's
Performance testing at the step, object, or script level (run-time of each)
-Track performance data over time
2
Smoke & Regression testing
2a
Modify (update, merge, delete) existing regression test scripts
Must Have
2b
Must Have
2c
Update existing regression test objects
i.e. id changed on a text field
Validation of Content (i.e. price, quantity, discounts, total price, promo discounts applied properly to estimated total)
2d
Depth of tests (just test 'happy path' or test every script you have for a particular functionality)
2e
2f
2g
-run deeper tests as needed
Turn a script "off" as needed
Tying scripts to application code - new changes, defects
1g
Must Have
Must Have
Nice to Have
Must Have
Must Have
Must Have
Must Have
Nice to Have
Nice to Have
Must Have
Must Have
Must Have
Must Have
Must Have
www.us.sogeti.com
Local Touch – Global Reach
15
16. Local Touch – Global Reach
Thank you
www.us.sogeti.com
