1. ALN DC Chapter Meeting
March 2012
Agile Testing for Mere Mortals
Presented by Dave Haeffner

2. ALL BOOKS OR SERVICES MENTIONED OR
RECOMMENDED ARE COOL AND WORTH SHARING.
DAVE IS NOT AFFILIATED WITH THEM IN ANY WAY
SHAPE OR FORM.
IF ANYONE HAS A MEDICAL CONDITION, OR IS
ALLERGIC TO FIRE, DON'T SIT IN THE FRONT ROW.
AND PLEASE, FEEL FREE TO ASK QUESTIONS
THROUGHOUT.

4. Show of hands -- How many of you are human
beings?
Keep them up. Ah, good we have our control.
Keep your hand up if you've had experience with
Agile Testing, QA… or Automated Web Testing.
OK. Keep your hand up if the thought of it leaves a
bad taste in your mouth.
Look around. You're not alone.
Thanks for coming out.

6. A little bit about me…
While working at The Motley Fool, I accidentally ended up in
this profession.
I used to be in IT Operations -- a Systems Administrator.
Through a twist of fate, I ended up trying out QA for a change
of pace. It was supposed to be temporary…
That was over 3 years ago…
When I started out, I had a lot of questions. A big one was --
"Ummmm, what is Agile Testing?”
I'd like to share with you some of what I've learned since
then. It should only take about 5 minutes.
And I'll deliver it in 3 parts so it's more digestable -- So I'll give
you the text book version, the "street" version, and
Automation.

8. I tend to leave Automation last because it's a good excuse to read you a quote. This
one's from Randy Pausch's "The Last Lecture":
"No, I did not make it to the National Football League, but I probably got more from
that dream and not accomplishing it than I got from any of the ones that I did
accomplish. I had a coach, I signed up when I was nine years old. I was the smallest
kid in the league, by far. And I had a coach, Jim Graham, who was six-foot-four, he
had played linebacker at Penn State. He was just this hulk of a guy and he was old
school. And I mean really old school. Like he thought the forward pass was a trick
play. So, we showed up for practice the first day, and you know, there’s big hulking
guy, and we were all scared to death of him. And he hadn’t brought any footballs.
One of the other kids said, excuse me coach, but there’s no football. And Coach
Graham said, right, how many men are on a football field at a time? Eleven on a
team, twenty-two total. And Coach Graham said, all right, and how many people
are touching the football at any given time? One of them. And he said, right, so
we’re going to work on what those other twenty-one guys are doing. And that’s a
really good story because it’s all about fundamentals. Fundamentals, fundamentals,
fundamentals. You’ve got to get the fundamentals down because otherwise the
fancy stuff isn’t going to work.”
Let's dig in.

10. Ah, Agile Testing -- The White Whale of the Agile
World. Everyone knows it exists, but it can be hell
on earth to catch it. If you're successful, the
benefits are *massive*. If you fail, then your
abilities as a team can be severely limited, even if
you don't know it.
So let's unpack the idea of "Agile Testing" and focus
on what it is, it's key elements, and the key players
involved in it's delivery. For this, I'm going to consult
"The Book"

12. Agile Testing -- what is it?
Here's a good definition -- "doing the simplest tests
possible to verify that a piece of functionality exists
or that the customers' quality standard (e.g.
performance) has been met”. Sounds simple
enough, but there's obviously more to it, or else
you wouldn't all be here.
I look at that definition as the outcome you want.
But you can't get there by dictating it from the top-
down. It requires more... finesse.

13. Team

14. Crispin & Gregory advocate that you should put it to
the team, letting them work together to come up
with a "software quality strategy". They go on to
abstract this concept further, saying you need to
keep stoking that initial fire, and evolve it into a
"Quality Philosophy" -- making software quality a
part of the team's DNA -- or else your efforts will
not be as effective.

15. ! Team

16. While a main tenant of Agile Testing is a whole team
focus, Crispin & Gregory really emphasize the
importance of the "Tester" or "QA” role (they use both
names interchangeably). Their primary responsibilities
are varied, and include:
* Seeing the big picture, looking at the application
more from a user or customer point of view
* Being an integral member of the customer team --
helping elicit requirements and examples, and helping
the customers express their requirements as tests
* Being embedded on the developer team
* Should look for unique ways to facilitate
communication

17. x^3

18. One tool for facilitating communication is The
Power of Three, or (since I was just in Costa Rica)
The Three Amigos. NEXT SLIDE

19. 3 amigos

20. It means that all discussions about a feature
need to have a Developer, a QA, and the Product
Owner present. And it is the responsibility of
each person to make sure that each group is
properly represented.

21. Value & Risk
Team Bridge

22. So if you follow their model and have successfully ignited a
passion for quality -- then you should have an autonomous
whole team approach to quality and someone on the team
working to help bridge the gap between tech & biz. If you
take this and keep it simple, focusing on value for the
business, then things start to get interesting.
(Here's a quote -- "while we have skills to identify test cases
beyond the 'happy path' we still need to start by making sure
the happy path works.")
Seems simple enough. Let's all do that.
While it sounds simple, it's not easy. Because let's be honest,
Agile Testing is a lot like the classic 1981 video game "The
Oregon Trail”

24. Think about it…
It's a long journey (takes time)
You think you know where you're heading -- greener pastures and
so on (everyone talks about it – a UTOPIA)
You try to make it while keeping your family together (whole
team)
Only to be attacked by natives along the way (funding, politics,
other priorities, technology woes)
And you'll likely end up dying of dissentary (give up and stick with
the status quo)
Let’s draw some parallels (above in parens)
But all joking aside, there are additional challenges that prevent
Agile Testing from working. Here are some more subtle ones

26. When dealing with building out QA, you’ll find you
need both a technical and analytical set of
resources. Such skills exist in a single person, but
it’s hard to find. And odds are, they won’t stay in
the QA role for long. Unless they’re 6’2” with
brown hair and has a coffee obsessions
An approach to work around this is, pick one, and
leverage other resources to fill in the gaps. (e.g.
what The Fool did) – makes staffing *a lot* easier

28. Need more QA’s and having trouble sourcing
external candidates?
Higher from w/in – Customer Service is an
excellent place
And why not? They know your product inside
and out and interact with your customers on a
regular basis

30. Throwing it over the wall *shudder*
The problem is that often times the feedback that
QA’s give to developers is out of band with their
work flow
And there’s often not enough QA’s to go around,
they are spread too thin, and not truly embedded
on the team
The short game - use a CI server
The long game – use a CI server and get the team
using ATDD and well written step definitions (more
on this later)

31. “The one thing worse than screwing up, is
screwing up and thinking you’re doing well.”
– J.B. Rainsberger

32. In antithesis of this, Dev's treat QA Regression as
a safety net w/o knowing what is actually covered,
QA's do the same for unit tests
I say we get rid of the words "regression" and
"smoke". I've found that they tend to mean
different things to different people

33. Effectiveness = Quality * Acceptance

34. But also, you need to tie behavioral and process
change back to value
That way you can motivate people in the right
direction w/o forcing it – autonomy will reign
supreme
Start with why, then sort out the how and what
This will help your effectiveness formula – have
you heard of this formula?
(Effectiveness = Quality * Acceptance)

36. QA as a gatekeeper & defects still getting out?
Try a Proper process

37. Pre
Often times, things get released
Stories reviewed , if issues found, that bypass the previous steps
kicked back to Devs
Story Discussion, Sprint Goals, This reinforces a passive acceptance
Happens more often that you think of the inefficiencies of our process
Sizing, Tasking, and a dash of
Definition of Done (depending on
Automated tests written here It’s time for a change
the team)
IDEATION Planning WIP QA Review Done Released EXISTENCE
Work’s done! Time to throw it over Business Owner review, if they find
the wall to be “QA’d” an issue – it gets kicked back to
Devs
Sometimes happens without QA
knowing – causing duplicative
effort and additional context
switching for Devs

38. Post
Create Acceptance Tests Can be done at the same time
Becomes the Specification Batches feedback for Dev
Magically becomes an automated web test Expl. Testing done by QA (or someone that didn’t work on the story)
Require rep’s. from each group (Biz Owner, Review done by Stakeholder
Dev, QA, UXD)
Shovel Exploratory
IDEATION Planning WIP Review Done Released EXISTENCE
Ready Testing
This is *the* most important PART
Automated Acceptance Tests need to pass Everyone can rest easy, because this
in order to proceed software is high quality baby!

40. My train of thought spawned from a
tremendously good read by Gojko Adzic where
he introduced me to the concept of ATDD
It solves so many problems and just makes so
much sense.
Different dialects from team to team & btwn biz
& tech? create a ubiquitous language
If you get this right, then you win the game, you
have completed the oregon trail and did not die
of dysentary

41. RED
Test Code
GREEN
REFACTOR
(repeat)
Commit
Given <precondition(s)>
When <action>
Then <result>

42. Automation frees
you to do more
exploratory testing!
Acceptance Test
Driven
Development
(ATDD) or BDD
Test Driven
Development
(TDD)
http://watirmelon.com/2012/01/31/introducing-the-software-testing-ice-cream-cone/

43. Not ready to automate? That’s fine.
If you start to capture new features in the gherkin
syntax, you will be heading in the right direction.
It will act as a communication and collaboration
tool between your team and the business.
And it will enable you to have a shared, ubiquitous
language that business and tech can both agree on
and understand
If you add automation later, then you get the
retroactive benefit of already captured features.

47. Traditionally w/ automated web testing using a
free tool like Selenium there were two paths
IDE -> big sweet -> slow -> brittle -> false
positives
IDE -> RC -> abstraction, logic, smarter ->
more technical (read “custom testing harness”)
And the big complaint among Devs is that it’s
too slow – they call it full stack testing since it
loads the browser to execute

49. The funny thing about Selenium
Jason Huggins built Se to solve a problem
much like Nike, then founded a company to
solve the problem he created.

50. By Aslak Hellesøy

51. Right around the same time, Cucumber came
about. It’s a ruby implementation of BDD. It enables
you to automate gherkin feature sets
Where Gojko evolved the approach of Crispin &
Gregory
Aslak provided automation for Gojko’s approach to
capturing features
And you can drive Selenium through Cucumber
And push it to the cloud, or run things parallel
locally

53. By taking the ATDD & Cucumber (or other BDD
implement) route, you short-circuit the entire
headache chain that has plagued Agile Testers that
have gone before you
Granted, there are new challenges, given that the
step definitions need to be crafted to fit the context
of your environment, and that is code. But the
benefits, IMO, greatly outweigh the cost

54. But but…

55. I often get the question – but what if we need to
move fast? Or have a legacy code base? We
don’t have time for this, what do you
recommend?

57. Focus on the 3 buckets
1) SETI
Are the foundational pillars of your system alive
and responding?
2) Money Makers
What in your app is responsible for directly
generating value to the business? Focus on that.
3) Back breakers
Watch out for that hole in the Death Star

59. Keep it simple
This advice is more for people who are
going to be writing the glue code
Focus on IDs -- XPath is a last resort
Hard to test apps are a symptom of poor
design

61. Take steps towards speed (and long term
sanity)
Make each test scenario fast – 1-2 min
Keep them autonomous (e.g. unique accounts,
set up built in, no cross dependencies, etc.)
Single assertions per scenarios
One interpretation of test results

63. The prior will make parallelization possible
Options abound for both local and cloud
offerings

65. But all that feedback is pretty useless unless
you do something with it
Build a feedback loop
Make sure that developers are getting
feedback in line with their work flow, else
hear the dreaded words "Works on my
machine"

66. A checklist for you
Make Quality The Whole Team’s responsibility
Tie things back to Why
Build a ubiquitous language
Promote from within
Keep it simple
Focus on the 3 buckets (SETI, Money Makers,
Back Breakers)
Parallelize for speed

67. @TourDeDave
http://www.slideshare.net/tourdedave