This document outlines the key components of creating a business case for test automation. It discusses justifying automation in terms of costs and benefits by estimating return on investment and time savings from automating test development, execution, and evaluation. The business case should identify the problem, propose an automated solution, and analyze benefits, costs, risks, and obtain management approval. Examples of needs that could justify automation include increasing efficiency, coverage, and handling tests not feasible manually.
2. Ways to Automate Testing
• Just do it
• Buy a tool and start using it
• Usually fails - Lack of skills, Applied to wrong
part of test process
• Perform analysis and create business case
• Prepare business case through proper
analysis
• Get management support and approval
3. Definition of the Business Case
• A document that proposes the solution to a
problem in business terms
• Identify the business need
• The problem to solve (e.g. can’t test effectively)
• The proposed solution (e.g. Automate)
• The benefits of proceeding (e.g. faster test, better
coverage, finding more defects)
• Include an initial cost analysis using ROI
• Identify potential risks and mitigation approaches
4. Business case contents
• Executive summary
• Description of problem/business need
• Solution recommendations for problem/need
• Cost and benefit justification
• Strategic alignment justification
• Initial assumption, constraint list and risk analysis
• Initial budget and resource estimates
• Approval signoffs
• …
5. Some example needs
• Speeding up the test effort, increasing efficiency
• Decreasing testing costs
• Using test engineer skills more effectively
• Increasing test coverage, including doing tests that
can’t really be done by hand
6. Justifying Automation in Terms of Cost
and Benefits
• Estimating ROI
• Developing automated tests requires more effort up
front than manually documenting a test
• Maintenance of these tests needs to be considered
• The payoff really comes from tests that are run on a
regular basis, increased test coverage or tests that
manual testing can hardly accomplish
• The automation of the test results evaluation can
realize time savings as big or bigger than those of the
actual test execution
7. Justifying Automation in Terms of Cost
and Benefits
• Overall Test Automation Savings
• Test Environment Setup Time Savings
• Test Development Time Savings
• Test Execution Time Savings
• Test Evaluation/Diagnostics Time Savings
8. Overall Test Automation Savings
Item Time Savings
(Hours)
Cost Savings (4)
Test development -250 -25,000
Test environment setup
Test execution 1,583.33 $158,333
Test evaluation/diagnostics 2,250 $225,000
Other -25,000
Total 3,583.33 $333,333
9. Test Environment Setup Time Savings
(per test phase)
Item Manual Test
Setup (Minutes)
Automated Test
Setup (Minutes)
Test setup time
Number of test iterations
Subtotal (setup time * iterations)
Other
Total setup time (subtotal + other)
Time saving (manual total – automated
total)
10. Test Development Time Savings
• Initially there will be an increase in time rather than
a savings due to the extra steps necessary to design,
create and test the test scripts (programs)
• This computation does not include the design and
implementation of the actual automated test
environment.
12. Test Development Time Savings
Item Manual Test
Development
Automated
Test
Development
Number of tests planned
Estimated time to develop a test
Total test development time
Difference between manual and automated
total times
13. Test Execution Time Savings (per test
phase)
Items Manual Test
Time
Automated
Test Time
Number of tests planned
Estimated execution time per test
Number of test iterations
Total test time
Difference between manual and automated
total times
14. Test Evaluation/Diagnostics Time
Savings
Item Manual
Evaluation
Automated
Evaluation
Number of test output entries to
evaluate/diagnose per test iteration
Estimated time to evaluate/diagnose a single
test output
Number of test iterations
Total test evaluate/diagnose time
Difference between manual and automated
times
15. Justifying Automation in Terms of Cost
and Benefits
• Other ROI Considerations
• Personnel requirements
• Lab equipment requirements
• Test case maintenance
16. Justifying Automation in Terms of Cost
and Benefits
• Intangible Savings (ROI)
• Reducing the number of test cases to a necessary
minimum
• Selecting the right test cases to cover all possible
scenarios
• More ROI Metrics
• Intangible benefits that cannot be expressed by ROI
• improved product quality
• improved customer satisfaction
• …
17. Risks
• Short time-to-market
• Lack of adequate skills
• New technology
• Difficult-to-automate requirements or features
18. Other Considerations
• How will your test team keep up with testing the
ever increasing complexity of new technologies?
• How will your test team’s productivity keep up with
the rate of productivity improvement expected from
the development team?
• What is your testing strategy to support the
increasing level of software and component reuse
that is planned?
• What artifacts can you provide to demonstrate the
software was tested to meet specific standards? How
quickly can those be produced and reproduced for
each test run?
19. Other Considerations
• How do you capture and reuse the knowledge of the
subject matter experts your test program currently
depends on?
• What is your plan for being able to document and
re-create problems found during testing for a
development that is geographically distributed?
• When the software product can be installed
remotely, what is your approach to providing
verification or diagnostic tests that can be executed
remotely?
20. Other Considerations
• How will you run tests that manual testing can
hardly accomplish, such as memory leak detection,
stress testing, performance testing and concurrency
testing?
• Will you need to justify having testers work night
shifts, weekends and overtime when an automated
test tool can be kicked off at day’s end and the
results viewed the next morning?
