The document discusses acceptance testing frameworks, including what they are, why they are worthwhile, and what should go into one. It defines an acceptance testing framework as one that defines the format for expressing expectations, drives the application under test, executes the tests, and reports results. Frameworks help avoid issues like brittle tests and performance problems that commonly occur with tools like Selenium. Effective frameworks have components like a runner, integration, configuration, reporting, logging, and a domain-specific language. The real value is realized by integrating frameworks with continuous integration to provide timely test results. Several open source and commercial frameworks are described.

1. Frameworks^3
by Dave Haeffner
ARRgyle.com

2. Frameworks Agenda
What it is
Why it's worth it
What should go into one
What's available today
Q&A

3. What is...
an acceptance testing framework?

4. Elizabeth Hendrickson says...
1. Defines the format in which to
express expectations
2. Creates a mechanism to hook into, or
drive, the application under test
3. Executes the tests
4. Reports the results
[link]

5. Why is...
an acceptance testing framework...
worthwhile?

6. The Standard Evolution of
Selenium automation at an organization

7. 1 - Record tests with Selenium IDE
2 - End up with a brittle suite of tests
3 - Export to code, get them working
4 - Rewrite them so they're DRY
5 - Scale, hit performance issues
6 - Monkey-patch as you go
A - Ignore it and go back to manual testing
B - Throw it out and rewrite
Standard Selenium Evolution

8. Hmm...

9. What really makes up...
an acceptance testing framework?
(especially one that uses Selenium)

10. Framework anatomy
Current
Runner
Integration
Configurable
Reporting
Logging
Performance
DSL
Taggable

11. But the real value isn't realized...
until you do something with it.

12. Why Feedback loop for the team
How Integrate with CI
What Timely, in-band notifications

13. Frameworks in the wild...

14. Open Source
Py.Saunter (Python)
SaunterPHP (PHP)
ChemistryKit (Ruby)
Geb (Groovy)
SST (Python)
ScalaTest (Scala & Java)
Baby Framework (Java)
JDriver (Java)

15. Open Source cont'd
Robot Framework (Python & Java)
Fitnesse (Java & .NET)
Cucumber (Ruby)
Cucumber-jvm (Java)
Specflow (.NET)
Turnip (Ruby)

16. Proof
Titanium xAFT
Commercial Offerings
SauceLab Partners
QAOnDemand
TFT Consulting
Dynacron Group
Softcrylic
ThinkSys Inc
Hindsight Software
QASource

17. Recap
● An 'acceptance testing framework' that is
built for successful Selenium has a lot of
parts
● A framework will help you get started, grow,
and avoid pitfalls
● There are many "frameworks" (both open
source and commercial)
● But there are few that come with everything
you need out of the box
● The real value comes from using it within
your development workflows

18. Office Hours (30 min, FREE)
sohelpful.me/tourdedave
Blog & Newsletter (FREE)
arrgyle.com
dave@arrgyle.com@TourDeDave

19. Additional Resources
Selenium HQ's outdated framework page
Agile Alliance Functional Testing Tools
spreadsheet (outdated-ish)
wiki (should be the record of truth)
List of available Selenium Resources
blog post on arrgyle.com
SF Selenium Meetup's Testing Tool Demo Night videos