Oleksandr Reminnyi, System Architect at SoftServe, Inc. compared seven projects for his presentation on Failure and Success in Software Automation at the 2013 IT Weekend Ukraine conference that took place on September 14, 2013, in Kyiv, Ukraine. Here`s Oleksandr`s presentation.

1. Failure and Success in Software
Automation
Comparing 7 Projects
Oleksandr Reminnyi

2. Pre word
▪ The only identification of project being
successful is the fact that your customer is
happy.
PMbook

3. Understanding the goals
We want to save money and
time later on manual testing
We have a corporate
direction…
But the main is…
Automation is a BUZZ word

4. Average Automation Reality

5. Automation AND Skill Set

6. HOKKU
«Let`s automate everything!», the
client said.
And Mount Fuji trembled.

7. Customer skills
Should customer be mature in automation?
This is definitely not a prerequisite.
But it can influence…

8. ХОККУ
Читаю чужі тести і торкається моїх
скронь сивина…
То мозок проливається

9. Automation Dev. Team Skills
Does mature mean good automation here?
Not always.
Practices mean more.

10. Automation AND Time line

11. Automation and SDLC
When the automation might be successful?
Waterfall OR Agile?
Depends on automation layer;
Post release period!

12. Implementation OF one TC
Depends on complexity
Depends on number of steps
Depends on tested application
instrumentation
Average 4 to 20 man hours. Happens 50

13. Automation Toolset

14. Automation and Backend
Unit Tests
Really? Why?
Component/integration testing
Good job!
Service layer automation
Definite place to automate!

15. Automation And TOOLS
Can tool influence the success of
automation?
YES!

16. Automation of Mobility
Can mobile platform automation be buried before birth?
YES!
Silk Mobile $5000 per dev. per year,
20-30% failure rate only because of the tool issues
MonkeyTalk, Ranorex, etc.
Build in the agent, then find issues
Appium
maybe light in the end of the tunnel. But later
Consider out of device testing!

17. Documentation

18. Can automation exist without
regression test Cases suite?
YES!
At least they say it’s successful…

19. Different types of manual test cases
being Automated
-
Functional
Regression
Acceptance
…
Benefit is brought at
ones
once by Smoke

20. Automation Metrics

21. HOKKU
Sacura`s leaves are destined to fall down.
Once again we`ll see the Test Report in red.
Not only leaves are down.

22. What is usually Calculated?
ROI?
Nothing!
Sometimes:
Failure Rate
Suite Run Duration

23. Given architecture…
SUT
Automation Tool
UI
Services
…
Other SUT facilities that can be hooked but not
needed for automation tool access
Reporting
Base classes
Layer
Framework utilities
Test scripts
Test scripts
UI Maps – facilities to access UI (or other services) through
automation tools API
Layer
Etc.
Loading configuration
Test scripts Libraries
Test scripts
Functional
Logging
Layer
DB Helpers
Test scripts
Test scripts
Test scripts
Facilities to hook into SUT
through AF
Multi - layered architecture
Sometimes joined
Layer

24. Gave us possibility to collect More
metrics

25. Something obvious

27. Questions?
“I see the testing standards that do not tell
me how to create a test worthy of running."
DeMarco