Seven Keys to Navigating Your Agile Testing Transition

ial

Presented by:
Bob Galen
Vel rs
Brought to you by:

340 Corporate Way, Suite Orange Park, FL 32073
888‐2
MD
AM Tutor
4/7/2014
8:30 AM

“Keys for Transitioning to Agile Testing”

ocity Partne

300,
68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

An agile methodologist, practitioner, and coach based in Cary, NC, Bob Galen helps
rship
.

Bob Galen
ersVelocity Partn
guide companies in their adoption of Scrum and other agile methodologies and
practices. Bob is a principal agile evangelist at Velocity Partners, a leading agile
nearshore development partner; president of RGCG; and frequent speaker on
software development, project management, software testing, and team leade
at conferences and professional groups. He is a Certified Scrum Coach, Certified
Scrum Product Owner, and an active member of the Agile and Scrum Alliances. In
2013 Bob published Scrum Product Ownership–Balancing Value from the Inside Out
Reach him at bob@rgalen.com.

1
Keys for Transitioning to Agile
TestingTesting
Myths & Realities from the Trenches
Bob Galen
President & Principal Consultant
RGCG, LLC
bob@rgalen.com
Introduction
Bob Galen
Independent Agile Coach (CSC) at RGCG, LLC
Principle Agile Evangelist at Velocity Partnersp g g y
Somewhere ‘north’ of 30 years overall experience ☺
Wide variety of technical stacks and business domains
Developer first, then Project Management / Leadership, then
Testing
Senior/Executive software development leadership for 20 years
Practicing formal agility since 2000
XP, Lean, Scrum, and Kanban experience
From Cary, North Carolina
Connect w/ me via LinkedIn and Twitter @bobgalen
Copyright © 2014 RGCG, LLC 3
Connect w/ me via LinkedIn and Twitter @bobgalen
Bias Disclaimer:
Agile is THE BEST Methodology
for Software Development…
However, NOT a Silver Bullet!

2
Outline – Myths & Realities
Introduction
1. Transforming your
Team
9. Developer to Tester
WorkflowTeam
2. Automation
3. Developers &
Automation
4. Developers Testing
5. Test Planning &
Scripts
6. Testing within the
Workflow
10. Managing Agile Testers
11. Test Metrics
12. Retrospectives – The
Secret Sauce
13. Continuous Improvement
14. The Customer
15 Agile Requirements The
g
Sprint
7. Exploratory Testing
8. Role of Testers
15. Agile Requirements – The
Product Backlog
3-Pillars of Agile
Testing & Quality

3
#1, Transforming your team
Myth: You need all programmers or highly technical
testers when you move to agile
Reality: A mix is best –
Manual, domain-centric and technical skills
S i / i ti kill
Some programming / scripting skills
Soft / collaborative skills
Reality: And throw out all of that Developer-to-Tester
ratio ‘stuff’.
#2, Automation
Myth: You need 100% automation to start
agile testingagile testing.
Reality: You simply need to have a strategy AND
doggedly pursue automation where it makes sense
Make it part of the Backlog and work it every sprint
Reality: There are some excellent Open Source tools
that supplement agile automation development
Reality: The Agile Test Automation Pyramid is the right
overall strategy

4
Agile Test Automation Pyramid
Mike Cohn; Lisa Crispin & Janet Gregory
http://behaviordrivendevelopment.wikispaces.com/Testing
Brainstorm…
Agile, Multi-tiered Automation
Get together in small groups of 4-6 to discuss
Take a few minutes and think about your current
automation approaches:
Tooling, approaches & strategies, strengths, weaknesses,
opportunities, maintenance challenges, future technology, etc.
What sorts of adjustments would you need to make to
take this approach?take this approach?
What would be the largest challenges in taking this
approach? How might you overcome them?
Do you “buy” the whole-team view to automation?

5
#3, Developers & Automation
Myth: QA designs, writes & runs all of the test
automationautomation
Reality: Everyone should be responsible for automation
Developers need to minimally attend to Unit Level
Participate in any framework or re-use development
Writing ‘glue’ code – fixtures, step files, etc.
Reality: It also extends into your Build & Continuous
Reality: It also extends into your Build & Continuous
Integration systems
All automation should be ‘wired’ into CI
Dashboards, trending, lava lamps, etc. for all to see…
#4, Developers Testing
Myth: Developers can’t test their own code—they’re not
independent enough nor skilled enough to do it properly.
Reality: We need to stop stereotyping team members,
their strengths and their abilities.
Developers can absolutely test their own code.
Some are better at it than others
Pair with them to help test appropriately

6
#5, Test Planning & Scripts
Myth: You don’t need to plan
(it just happens )(it just happens…)
and you don’t need functional test cases
(automation takes care of everything…)
Reality: Plans help the team focus on the risk-based
testing required within an iteration AND across a release
Reality: Scripts (test cases) help formalize and drive
your testing;
Absolutely required in regulatory environments
Reality: You’ll never actually automate every test
Brainstorm…
Agile Planning & Execution
Get together in small groups of 4-6
Take a few minutes discuss your current planning and
test process mechanisms.
What would an Agile Test Plan “look like” in your organization?
What would Test Cases “look like”? What about progress
measures? And traceability?
Can you move from the “individual” to the “team”?Can you move from the individual to the team ?
Be prepared to share…

7
#6, Testing within the Sprint
Myth: You simply need to run 100%
of the tests within the constraints of the Sprint…that’s
“Agile”
Reality: Rarely possible in most contexts.
You first need a high-degree of automation and business support
(for example: equipment costs)
Very mature test automation and CI / CD environments
Reality: Most agile teams adopt some sort of risk-based
testing approach for within the sprints
Dealing with Technical Test Debt
Then leverage Hardening / Stabilization pre-release sprints
The Agile Release Train
Synchronized
Internal
Release
External
Release
Iterate
Iterate
Team 1
Team 2
Team 3
Iterate
Iterate
Harden Iterate Iterate Iterate
X-team
Harden
Harden
HardenIterate Iterate
Iterate Iterate
Iterate Iterate
Iterate
Iterate
Docs,
Training,
Support,
UAT,
Comp.
Continuous Integration
Copyright © 2014 RGCG, LLC
Team 4 HardenIterate Iterate Iterate Iterate Iterate
Team n
…
16

8
Example: eCommerce / SaaS Model
External
Release
10 days 10 days 5 + 2 days
Iterate
Iterate
Team 1
Team 2
Team 3
Iterate
Iterate
Harden
X-team
Harden
Docs,
Training
Harden
HardenIterate Iterate
Rinse
&
Repeat
Team 4 HardenIterate IterateTeam 8
…
Environment
Evolution Dev + QA Dev + QA QA -> Staging Production
17
Example: iContact / SaaS Model
Production
Release
3 weeks / 15 days 4-5 days
SBET, Exploratory –
Regression Testing
Team 1
Team 2
Team 3
Iterate
Iterate
Harden
X-team
Harden
Docs,
Training
Harden
HardenIterate
Rinse
&
Repeat
Team 4 HardenIterateTeam 10
…
Environment
Evolution Dev + QA QA -> Staging Production

9
Brainstorm…
“Your” Agile Release Train
Take a few minutes and think about your current release
constraints
timing, customers, domain, competition, # of teams, technology,
etc.
Design a release train model for your organization
O l it ith t ti ti iti l d il tOverlay it with testing activities, plans, and milestones
Present it to your larger table/group; gain feedback &
adjust
#7, Exploratory Testing
Myth: There is no place for Session Based ExploratoryMyth: There is no place for Session Based Exploratory
Testing in agile contexts.
Reality: ET and SBET are a beautiful complement to
agile testing.
Helping nurture pairing & collaboration across teams and
functionsfunctions
Defining new (more valuable) test cases
Quickly gaining quality & usability feedback
Let’s explore the details of SBET…

10
#8, Role of Testers
Myth: That the testers alone own quality & testingMyth: That the testers alone own quality & testing
practices within each team and sprint
Reality: The testers foster a “Whole Team” view
towards quality—focusing less on “Testing” and more on
“Quality Practices & the Customer”
S i id f th t T ti th “h d bit ”
Serving as guides for the team; Testing the “hard bits”
Facilitating exploratory testing sessions—finding more
interesting / valuable tests
Working with the Product Owners—are we solving the
customers problems?
#9, Developer to Tester Workflow
Myth: There is always a hand off from developers toMyth: There is always a hand-off from developers to
testers; usually quite late in the sprint. That’s simply the
“way of things” in software development.
Reality: Scrummer-fall is alive and well…but, Wrong!
Teams need to swarm on their work, as flow &
throughput matter the most
throughput matter the most.
WIP limits and close proximity / collaboration help establish a
healthy tempo of developer & tester pairing
Micro-handoffs – testing as development progresses!
Do you log bugs? Or do you fix bugs?

11
#10, Managing Agile Testers
Myth: The functional test manager is
in charge of deciding how who when etc for the testin charge of deciding how, who, when , etc. for the test
team.
Reality: You still absolutely need functional leadership
within agile teams;
However, it’s focused towards quality practices, strategy &
hi d h dli i di t / l ti
coaching, and handling impediments / escalations
Encouraging transparency, transforming metrics & reporting
Supporting & protecting the teams
Encouraging risk-taking, innovation & creativity (Slack Time)
Levels of Done-Ness Criteria
Activity Criteria Example
Basic Team
Work Products
Done’ness criteria Pairing or pair inspections of code prior to check-in; or
development, execution and passing of unit tests.
User Story or
Theme Level
Acceptance Tests
Development of FitNesse based acceptance tests with the
customer AND their successful execution and passing.
Developed toward individual stories and/or themes for sets
of stories.
Sprint or
Iteration Level
Done’ness criteria Defining a Sprint Goal that clarifies the feature
development and all external dependencies associcated with
a sprint.
Release Level Release criteria
Defining a broad set of conditions (artifacts, testing
activities or coverage levels, results/metrics, collaboration
with other groups, meeting compliance levels, etc.) that IF
MET would mean the release could occur.
24Copyright © 2014 RGCG, LLC

12
Brainstorm…
“Your” Definition of Done
Using the 4-tier approach referenced start filling in the 4
levels as a group.
Consider any criteria you are currently using at your
companies?
Also consider current issues or challenge you might
have where “done-ness” would help?
And what about Ready-ness?
#11, Test Metrics
Myth: You can and should move
forward reporting everything exactly asforward reporting everything exactly as
you have before.
Including any ‘dysfunctional’ metrics that your process and/or
PMO dictates.
Reality: The metrics should change immediately.
F QA d T t t i t d T C t i t i (V l
From QA and Test centric towards Team-Centric metrics (Value,
Throughput, Quality, Team)
Stop reporting out on “Testing”; it’s irrelevant!
This effects planning as well—estimation, progress, risk, etc.
Contribute quality-centric Information radiators to the mix

13
Brainstorm…
Morphing your Metrics
What are you measuring today? Why?
How are they driving your success and behaviors?
As you move to agile, what can/should you be
measuring in the 4 key areas:
Value, Quality, Throughput & Predictability, and Team?
How will you change your existing metrics? What
behaviors are you trying to inspire?
#12, Retrospectives:
The Secret Sauce
Myth: Testers are “Second Class” citizens who don’tMyth: Testers are Second Class citizens who don t
play an active part in the project & team
Reality: There are many places to “make a difference”
Getting the 800 lb. Gorillas out on the table; Showing courage;
telling truth
Fostering continuous improvement within the teamFostering continuous improvement within the team
Setting the example; showing vulnerability—admitting you’re
wrong
Team listening; active planning; dependencies; pairing
Risk-taking; Failure!

14
#13, Continuous Improvement
Myth: We’re generally ‘stuck’ in our
approaches so just accept them and doapproaches so just accept them and do
the “best you can”.
Reality: Continuous improvement is everyone’s
responsibility—to engage, suggest, take ownership of
current results, explore root causes, etc.
Active participation in your teams Retrospectives is a key way to
guide quality, testing, and customer-centric improvements.
Courage!
#14, The Customer
Myth: Business Analysts capture
customer requirements and testers test them forcustomer requirements and testers test them for
completeness.
Reality: You need to begin to partner with the Customer
– Stakeholders – Product Owners to produce software
that solves the their problems.
Move to the “front” and help define & refine User Stories with your
Product Owner
Actively participate in Sprint Reviews
Show value for automation; placing test investments in the
Backlog

15
#15, Agile Requirements –
The Product Backlog
Myth: We can’t start testing until the requirements areMyth: We can t start testing until the requirements are
finished or stable; no matter how ‘agile’ we are.
Reality: Hogwash! Get over it…
Ambiguity and incompleteness need to become your friend and
ally.
As does working with your Product Owners and Customers to
As does working with your Product Owners and Customers to
help define the requirements
Realizing that the requirements (User Stories) are only complete
at the end of each sprint.
Brainstorm…
Agile Requirements
Are iterative, are intentionally incomplete
The “older” the are, the larger and less defined they are
Enter the sprint at 70%, exit at 100%
Drive questions, dialogue, discussion, and collaboration;
think 3 Amigos or the Triad
So, WHY? And how will you make this work as a tester?

16
Agile Test Transformation Strategy:
3 Pillars of Agile Quality
Development & Test
Automation
• Pyramid-based Strategy:
(Unit + Cucumber +
Software Testing
• Risk-based testing:
Functional & Non-Functional
Cross-Functional Team
Practices
• Team-based Pairing
Selenium)
• Continuous Integration
• Attack technical
infrastructure in the Backlog
• Visual Feedback –
Dashboards
• Actively practice ATDD and
BDD
• Test planning @ Release &
Sprint levels
• Exploratory Testing
• Standards – checklists,
templates, repositories
• Balance across manual,
exploratory & automation
• Stop-the-Line Mindset
• Code Reviews & Standards
• Active Done-Ness
• Aggressive Refactoring of
Technical Debt
• User Stories, “3 Amigo”
based Conversations
34
• Whole Team Ownership of “Quality”
• Building it ‘Right’; Building the ‘Right’ Thing
• Healthy – Agile Centric Metrics
• Center of Excellence or Community of Practice
• Strategic balance across 3 Pillars; Assessment, Recalibration, and Continuous Improvement

17
Foundation of the 3 Pillars
• Whole Team Ownership of
“Quality”
• Whole team view includes building it right,
everyone tests,
• Focus on features/stories, confirmation,
i d i h d• Building it ‘Right’; Building
the ‘Right’ Thing
• Healthy – Agile Centric
Metrics
• Center of Excellence or
Community of Practice
conversation, and getting them staged
properly OVER testing
• 4-style metrics???
• Agile strategies need light-handed “steering”;
establish a CoE (heavier weight) or a CoP
(lightweight)
• Consider finding an assessment framework
35
• Strategic balance across 3
Pillars; Assessment,
Recalibration, and
Continuous Improvement
• Consider finding an assessment framework
and then tying it to your strategy
measurement, recalibration, and continuous
improvement.
• Make the Foundation visible thru Information
Radiators and metrics
Development &
Test Automation A central part of agile adoption is focusing on CI, 3-
tiered Automation development, and Dashboards to
b i i t ll b ildi f f t• Pyramid-based
Strategy: (Unit +
Cucumber + Selenium)
• Continuous Integration
• Attack technical
infrastructure in the
Backlog
begin incrementally building coverage for faster
feedback on changes.
In the interim, Hardening or Stabilization Sprints and
having a risk-based Release Train concept help
It’s important that Test or QA not ‘own’ the tooling or
all of the automation efforts. The strategy can come
from Test, but the tactical automation development is
best left to the team
36
• Visual Feedback –
Dashboards
• Actively practice ATDD
and BDD
best left to the team.
Mature teams invest in automation as part of Done-
ness and continually on their backlogs

18
Software Testing
• Risk based testing:
Exploratory Testing (Charter / Session based and
paired) can be an incredibly effective way to
establish a whole-team, collaborative view towards
quality and testing. It also emerges new tests.
• Risk-based testing:
Functional & Non-
Functional
• Test planning @
Release & Sprint levels
• Exploratory Testing
• Standards – checklists,
Leverage ‘plans’ as a whole-team collaboration
mechanism…and do plan.
Do not measure testing or tester progress; instead,
measure throughput, output, sprint outcomes, and
done-ness escapes at a team level.
You need a balanced test team; not everyone needs
to be able to program But everyone needs to be
37
,
templates, repositories
• Balance across
manual, exploratory &
automation
to be able to program. But everyone needs to be
skilled testers.
Agile testing is a Risk-Based play in every Sprint and
across a release sequence. Don’t forget your
techniques!
Cross-Functional
Team Practices
One of the hardest areas to get ‘right’ culturally. It
needs leadership alignment from Quality/Testing to
Product to Development and a consistent voice of
h l t h
• Team-based Pairing
• Stop-the-Line Mindset
• Code Reviews &
Standards
• Active Done-Ness
whole-team approaches.
This is where LEAN lives, where whole-team
collaboration happens, where professionalism and
craftsmanship are held dear.
I like the view of testers becoming the VOC,
champions of quality, and consistent questioners of
what is being build. Are we solving the right
problems as simply as possible Notions of Minimal
38
• Aggressive Refactoring
of Technical Debt
• User Stories – 3 Amigo
based Conversations
problems…as simply as possible. Notions of Minimal
Viable Product / Feature help with focus.
And yes Virginia, there ARE standards, templates,
and a focus on consistency!

19
Software Testing
Strategies
It ALL starts with empowering testers AND creating a
Whole-Team view towards QualityWhole Team view towards Quality
Critical Early Steps:
Creating a sense of empowered Functional Team
Applying Testing Standards across all teams
Deploying Exploratory Testing across all teams
Defining a core set of Agile KPI / metrics
Defining a core set of Agile KPI / metrics
ACTIVE participants in Sprint Planning
Cross-Functional Team Practices
Strategies
Training
Agile / Lean in general Story writing Acceptance Unit testingAgile / Lean in general, Story writing, Acceptance, Unit testing,
etc.
Teaming – for example: feedback or 5 Dysfunctions / Trust
Critical Early Steps:
Coaches & Scrum Masters to reinforce: Pairing / Swarming; WIP
Limits across teams
Define prescriptive and aggressive Done-Ness for ALL teams
Implement coding standards & Crucible / code reviews across the
center (appropriate for technology stacks)
Release Planning BEFORE allowing a team to start Sprint #1
Backlogs have Bug + Refactoring + Automation targets (20%)?

20
Organizational Quality
Strategies
Continuously communicate your unified Vision
Your strategy must be aligned/shared across:
Development, Quality/Testing, and Product
Keep working your strategy across the pillars
Don’t get stuck with too narrow a focus (easy road)
Make your strategy visible (Information Radiators)
Show progress (Ex: burn up of test automation coverage…across tiers)
Vi li i ti l i di t t A il Q lit
Visualize organizational impediments to your Agile Quality
strategies
Attack them!
Quarterly read-outs on progress, plans and adjustments
Listen to your teams; Celebrate successes!
What will be (your) agile strategy when you get
back home?
Either in groups or individually
Consider the 3 Pillars discussionConsider the 3-Pillars discussion
Consider your current team / organization agile ‘state’
Define a broad, 3-pillar view towards some immediate
focus points when you get back into the office
What will you focus on? Why?
How will you communicate the need for change?
How will you measure results?
What will come immediately afterwards?

21
Wrapping up…
Agile is the best thing that’s
happened to testers since…pp
The Great Depression
Whole Team view
Testing, Metrics, Automation
Planning, Reporting, Quality
Facilitate feedback
Multi-tiered automation
Just-in-Time risk-based testingJust in Time, risk based testing
Continuous improvement
Trust the Team
Retrospective
Contact Info
Bob Galen
Principal Consultant,
RGalen Consulting Group, L.L.C.
Experience-driven agile focused training,
coaching & consulting
Cell: (919) 272-0719
bob@rgalen.com www.rgalen.com
bgalen@velocitypartners.net www.velocitypartners.net
BlogsBlogs
Project Times - http://www.projecttimes.com/robert-galen/
BA Times - http://www.batimes.com/robert-galen/
Podcast on all things ‘agile’ - http://www.meta-cast.com/
44Copyright © 2014 RGCG, LLC 44

22
Additional Topics
Two Pillars of Lean ‘Thinking’
Respect for
P l
Continuous
ImprovementPeople
Customer, Employees,
Vendors…
Develop your teams
Trust & coach
N t f l k
Improvement
Embrace change,
challenge everything
Kaizen – small, incremental
change
Kaikaku – larger scale,
No wasteful work
Kaikaku larger scale,
fundamental
46
From http://www.leanprimer.com

23
Agile Testing Quadrants
Brian Marick; Lisa Crispin & Janet Gregory
Exploratory testing
S i
Functional tests
St t t
Automated &
Manual
Manual
Business Facing
Scenarios
Usability testing
UAT
Alpha / Beta
Unit tests
Component tests
Story tests
Examples
Prototypes
Simulations
Performance testing
Load testing
Q1
Q2 Q3
Q4
SupportingtheTeam
CritiquetheProduct
Component tests
API tests
Load testing
Security testing
Non-functional requirements
Automated &
Manual
Automation,
Tools, and
ManualTechnology Facing
10 Tenets of Agile Testing Jean Tabaka, Rally Software
1. The system always runs
2. Stop the line, vs. logging
Lean – fix it now!p gg g
defects
3. If it’s not tested, it’s not
“Done”
4. Testing comes first, not last
Early feedback; Earned
Value
Collaborative testing, focus
on building in quality
5. Finding defects after
Development is “Done” is too
late
g q y
Early feedback; fix it now!

24
10 Tenets of Agile Testing Jean Tabaka, Rally Software
6. “Development Complete” is
meaningless
Whole Team complete view
– no “partial credit”
7. Use testing, not analysis, to
explore requirements
8. Automation is “how” not a
“whether” or “when”
9 Tests are your second most
Executable requirements
Automate all testing;
feedback
Code is first; later is
9. Tests are your second most
detailed specification
10. Testers are Customer-
Developer liaisons
traditional specifications
VOC; guide effective team
collaboration; ask the right
questions
10 Commitments of Agile Testing Jean Tabaka,
Rally Software
1. We commit to not moving forward if a hole is found
through root cause analysis without first writing a testthrough root cause analysis without first writing a test
2. We commit to not relying solely on just automated
testing or just manual testing
3. We commit to not sitting behind a QA wall (no
boundaries!)
4. We commit to not allowing a code complete without
test code harness complete
test code harness complete
5. We commit to not waiting for a test phase but rather
working in smaller and smaller pieces, sooner and
sooner

25
10 Commitments of Agile Testing Jean Tabaka,
Rally Software
6 We commit to not testing one iteration after6. We commit to not testing one iteration after
development is “Done”
7. We commit to not allowing surprises to accumulate for
large end-to-end testing (“mock it now”)
8. We commit to not leaving the riskiest tests to the end
9. We commit to being an equal participant with the
customer and the developer in defining “Doneness”
10.We commit to not taking this oath lightly

Seven Keys to Navigating Your Agile Testing Transition

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (16)

Similar to Seven Keys to Navigating Your Agile Testing Transition

Similar to Seven Keys to Navigating Your Agile Testing Transition (20)

More from TechWell

More from TechWell (20)

Recently uploaded

Recently uploaded (20)

Seven Keys to Navigating Your Agile Testing Transition