Sustainable Automation
Frameworks
© 2016 CoverMyMeds LLC. All Rights Reserved.

Time to Get Serious
© 2016 CoverMyMeds LLC. All Rights Reserved.
• The ultimate goal of testing is to allow our products to reach
production faster and safer.
• We can’t do this if we’re sinking time into maintaining our
test suite.
• This helps no one.
• Our automated test framework should be lightweight and
easy to resolve problems.
• This allows us to get back to the more important task of
actually ensuring the quality of our applications.

How do we get there?
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Page-object model
• Step definition reusability
• Data management
• Other topics as my abstract promised

The Problem: Hardcoding
© 2016 CoverMyMeds LLC. All Rights Reserved.
• There’s a lot of STUFF in the THING you’re testing
• Lots of pages and objects on a web app
• Lots of services in an API
• Straight out of the box Google, most frameworks don’t have
a good way to manage this STUFF
• Then, when something changes, there is no one central
repository to update and we wind up having to make fixes
EVERYWHERE

The Solution: Page-Object Model
© 2016 CoverMyMeds LLC. All Rights Reserved.
• So we call it page-object… but really it’s applying object
orientation
• For each logical group we’re testing, we have a class that
describes it
• This works for web apps
• One page, one class
• Or for APIs
• One service, one class

What does this do for us?
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Makes our code reusable
• We can even package it into a gem and share with other
areas
• Makes our code easy to update
• One change can fix every failing test
• Makes our code easy to understand
• The code now reflects the system that is under test in a
logical manner

The Problem: Too Many Step Definitions
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Someone creates some step definitions that do the thing.
• Someone else creates another step definitions to do a thing.
• Nobody checks to see if there’s an existing step that can do
the thing.
• Everyone just keeps adding more step definitions until
there’s too many for a reasonable human being to search
through.

Why does this happen?
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Step definitions aren’t written to do what they say.
• Sure, there’s a step that says it does that… but it only
partially does it. Or does some other thing. And it’s not
what you actually need.
• Step definitions are unclear.
• You could spend an hour figuring out what that thing is
supposed to be doing… or you could spend ten minutes
writing your own step.
• No one polices the step definitions.
• Because we all really hate maintaining our test
framework.

The Solution: No One Quick Fix
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Standards
• Have a method of organization for your library
• Define at what level you want your steps to be written
• Everyone has to follow these!
• Step definitions ARE NOT CODE
• Structuring step definitions to represent coding logic
results in many incomprehensible steps
• Write step definitions to represent higher level business
logic

The Solution: No One Quick Fix
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Review your step definition library regularly
• This should be a quick check that can be done as part of
the normal process of writing tests
• If you’re having to make time or use special tools to
manage your library… your library is probably too big
• Get it under control and keep it under control
• Abstract your steps
• Don’t make your steps dependent on data
• Instead, let data drive the test

The Problem: Data Management
© 2016 CoverMyMeds LLC. All Rights Reserved.
• If your framework consists of objects and your steps are
flexible and business-oriented… data becomes the driver for
testing
• Data is no longer locked in as part of the step definition, so
we can load different data for each test
• So how do we manage large amounts of data?

The Solution
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Have a consistent, easy-to-use way of loading data into
your test
• Helper methods?
• Dump it into a fixture in a hook?
• Use descriptive naming for your data sets
• Reuse data where possible
• Make default data for stuff you don’t care about
• Creating a new test becomes as simple as changing the
data around

How to Keep it Pretty
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Be disciplined
• Don’t put off tomorrow what you should really be doing
today
• Have a hierarchy of changes
• Change the data
• Change the gherkin
• Change the step definition
• Change the page-object

How to Keep it Pretty
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Know what’s in your test suite and trust it to do it’s job
• Use all the resources available
• Don’t write unit tests with a GUI-driven test suite
• Not everything needs an automated test