3. Agenda
• Background - Why test automation, common
view of test automation?
• Define Problems - Issues most organizations
face with test automation.
• Fixing the above issues with architecture that
has the most ROI.
• Review
4. Why Test Automation?
Dev Team’s Perspective
• Increase effectiveness,
efficiency, and coverage of
software testing.
• For regression testing.
• Improve quality.
Boss’s Perspective
• Reduce operational cost.
• Increase revenue.
5. What comes to your mind when
you hear the term “We have
Automated Tests”?
8. Focused on GUI testing
• Hire consultants onshore/offshore with
mantra “Automate everything”.
• Qtp, Watir, Selenium Webdriver etc. use tools
specific to GUI tests.
• Every single small scenario gets added to GUI
automation.
10. When you need the test to run - It
never works.
Please work!
11. What’s the problem eh?
• Test execution is slow.
• Tests are fragile.
• Tests are not reusable.
• Tests needs more infrastructure.
• Test never works on my box but
fails on daily run.
More time spent on figuring why
the tests are failing.
Resource tied up just reviewing
failures.
Maintenance takes way too
much time.
17. GUI Layer test
• Test the whole application end to end.
• Sensitive to UI changes.
• Does not require good code.
• Works for Legacy system and with newer software
architecture like SOA as it is independent of backend
technology.
e.g. Open a browser, fill in the form and submit the data.
18. API/Service Layer test
• Test the application logic by making service calls.
• Faster than UI testing.
• Requires service to exist.
• E.g. Instantiate the calculator service and get it to
add two numbers.
19. Unit level test
• Test individual classes.
• Much faster than service level testing.
• Requires good design.
20. Application UI
Tightly Coupled
Automated UI tests
Service LayerAutomated One Level
Below UI
DAO Layer
Fragile
Lag with Current Development
Robust & Stable
With Current Dev cycle
Fast Execution
UI Independent
22. Start with Unit Test
(Quality is not just QA’s responsibility)
• Isolation.
• Test small piece of code - preferably separate test for
each method.
• Biggest test should be one test for a class.
• Use mock services to remove dependency.
• Test only public endpoints.
• This forces to use good design.
23. Invest more in Service Layer Test
• Independent functional test.
• Use BDD tools like Cucumber, SpecFlow,
Jbehave.
• Use mocks but also have specific tests to
check third party calls.
• Add data validation tests.
• Tests for boundary condition.
• Tests for error handling - wrong data
type, missing data.
• Tests around calculations, business rules
etc.
Gherkin/Plain test
DSL(Domain Specific
Language)
Set up
projects/helpers,
mock
Framework
Nunit,Junit,Rspec,
xunit
GUI
Service Layer
24. Limit the Number of GUI test
• Ask if it can be tested in Service layer before starting.
• Start out with use and throw style test.
• Gradually build the framework.
• Use GUI test as an assistance for manual testing.
• Create fewer tests but with broader end to end
coverage.
25. Quick Guide to Building GUI Test
Framework
• Do basic meta - programming and start
automation. Don’t design the entire
framework right away.
• Re - factor often.
• Review test automation architecture
regularly.
• Run daily.
Gherkin/Plain test
Framework
Watir/Selenium
Webdriver/QTP
WorkFlows
Pages
Navigation
UI
Utilities
26. GUI
Service/Api
Layer
Unit Test
Unit Test
Business Logic
Data access
layer
Mocked Data
access layer
Mocked
Database
Unit tests that isolate
dependencies with fakes,
mocks, stubs etc
Database
Can use just unit test
framework like Nunit,
Xunit prefer BDD style
like Cucumber, specflow ,
Jbehave etc
End to end UI
test with tools
like Watir,
Selenium, QTP
etc
Automated UI
Test
UI
Unit Test
Integration
Test
Integration
Test
Integration
Test
Business Logic Business Logic
Data access
layer
Data access
layer
Data access
layer
Mocked
Database
Database
Database
Data access
layer
Business Logic
28. Review
• Prioritize Unit and API/Service test
automation.
• Limit the number of GUI tests, have
a fixed number of stable end-to-end
tests.
• It’s ok to use multiple technologies.
• It’s ok to use Record/Play tools like
Selenium IDE for throwaway tests.
• It is better to have 50 stable test
cases than 500 fragile ones that
breaks regularly.
29. Hope you all catch tons
of Bugs!!
https://www.linkedin.com/in/bhupeshdahal
Editor's Notes
Idea being sold
We forget or ignore fundamental
Investing just on GUI test
Defined by Mike Cohn in succeeding with agile
Lets go ahead and define each layer
This provides many advantage of end to end testing without issue of UI tests. The service layer allows for testing business logic at the API or service level, where you're not encumbered by the user interface (UI).
Summary of above definition
This provides many advantage of end to end testing without issue of UI tests. The service layer allows for testing business logic at the API or service level, where you're not encumbered by the user interface (UI). DSL for setting values as per business criteria
Your focus now at the UI level is simply to ensure that the UI itself is working correctly. UI tests are very brittle, so keep these tests to a minimum. These automation tests will need maintenance any time the UI changes, and because there are so many factors that come into play when you run a test that emulates clicks on a screen (such as network speed), such tests can result in false test failures. You can't ignore those test failures, but you don't want to end up spending more time troubleshooting and maintaining UI tests than you spend finding actual code defe