As online activities create more revenue, organizations are turning to Selenium to test their web applications and to reduce costs. Since Selenium is open source, there is no licensing fee. However, as with purchased tools, the same automation challenges remain, and users do not have formal support and maintenance. Proper strategic planning and use of advanced automation concepts are musts to ensure successful Selenium automation efforts. Sharing his experience designing and implementing advanced automation frameworks using Selenium WebDriver, David Dang describes the factors necessary to ensure open source automation is right for your project. David helps you understand the real effort required to implement WebDriver in a way that will scale and minimize script development. Additionally, he dives into must-haves in your Selenium framework design; the resource and timeline considerations necessary to implement WebDriver; and the long-term, continual improvement enhancements all automation engineers should consider in their Selenium automation implementations.

1. 4/27/2015
1
Slide 1 Software Peace of Mind in Action
Presented By: David Dang
Leveraging Open Source Test Automation: A
Selenium WebDriver Example
StarEast 2015
Slide 2 Software Peace of Mind in Action
???
Introduction
Case Study Background
Decision to select open source vs. packaged tool
Assess and plan for test automation
Design an automation framework for Selenium
WebDriver
Framework Implementation
Outcome
Conclusion

2. 4/27/2015
2
Slide 3 Software Peace of Mind in Action
???
Most companies grasp the importance of leveraging test
automation to increase test coverage, reduce execution
time, and provide more frequent testing. But test
automation can be expensive. More than ever,
companies are seeking open source tools like Selenium
to reduce the cost. This case study demonstrates one
company’s experience using Selenium for test
automation.
Introduction
Slide 4 Software Peace of Mind in Action
???
Software company that provides online instructional
tools for teachers and students
Agile software development methodologies used
Continuous integration deployed
Company open-minded toward open source toolsets
Good collaboration between QA and development
Automated unit testing paired with Continuous
Integration tool
Case Study Background

3. 4/27/2015
3
Slide 5 Software Peace of Mind in Action
???
Needed a robust automation suite to perform “UI”
testing for integration/regression
Needed a subset of the automation suite (smoke test) to
run as part of Continuous Integration
Needed method for product owner, developers, and QA
to create automated tests
Needed to ensure maintainability of automated suite
Needed the automated suite to handle advanced and
dynamic web content
Case Study Background
Slide 6 Software Peace of Mind in Action
???
Selenium - Positives
FREE!
Supports multiple browsers
Scripts can be created with multiple programming
languages (Java, Ruby, Python, etc…)
Runs on Windows, Linux, and Mac
Not confined to the features of the tool
Integrates with other tools
Selenium vs. HP UFT

4. 4/27/2015
4
Slide 7 Software Peace of Mind in Action
???
Selenium - Downsides
Only supports web-based natively
No formal support (the user community provides
good feedback but questions are not always
answered)
Higher technical skillsets necessary to optimize the
Selenium framework
Reporting features are not as robust
Harder to find resources with both QA/testing and
Selenium experience
Selenium vs. HP UFT
Slide 8 Software Peace of Mind in Action
???
UFT - Positives
Established tool with wide usage
Supports multiple technologies (Web, Java, .Net,
SAP, Siebel CRM, PeopleSoft, Terminal Emulator,
etc.)
Has many built-in features (embedded data table,
smart identification, object configuration, object
repository, etc.)
Ease of use
Professional support, although room for improvement
Selenium vs. HP UFT

5. 4/27/2015
5
Slide 9 Software Peace of Mind in Action
???
UFT - Downsides
Cost! Purchase cost and yearly maintenance fees
Only supports IE and Firefox
Cannot run multiple instances on one machine
Selenium vs. HP UFT
Slide 10 Software Peace of Mind in Action
???
Cost
Alignment with the organization direction (most of the
development toolsets are open source)
Alignment with Agile methodologies
Ability to test on Mac
Ability to test on multiple browsers
Perception by both development and QA that Selenium
was easier to adapt
Factors in choosing Selenium

6. 4/27/2015
6
Slide 11 Software Peace of Mind in Action
???
QA roles and responsibilities
Type of testing within a sprint (unit, functional,
integration, etc.)
Test environments
Test data strategies
Level of testing (UI, backend, database, etc.)
Application technologies
Defect tracking
QA resource skillsets
Assess QA Process
Slide 12 Software Peace of Mind in Action
???
Develop a test automation framework/approach based
on findings and recomendations from the assessment
Plan and Design Test Automation

7. 4/27/2015
7
Slide 13 Software Peace of Mind in Action
???
Determine the number of resources necessary to build
the framework
1 Lead and 3 automation developers
Determine the timeline to completely build out the
framework
5 months
Identify Selenium components, development language,
and IDE
Selenium WebDriver, Ruby, Eclipse
Plan and Design Test Automation
Slide 14 Software Peace of Mind in Action
???
Identify execution environment (physical machines,
VMs, or cloud services)
Cloud service
Identify long-term maintenance and execution resources
The automation engine is handled by a dedicated automation
specialist
The automated tests are supported by QA or development
Determine automation development methodologies
Scrum with 3 weeks sprint
Determine training and mentoring needs
Plan and Design Test Automation

8. 4/27/2015
8
Slide 15 Software Peace of Mind in Action
???
Build the components:
Data file
Object map
Engine components
Logging file
Reporting engine
Test Automation Implementation
Slide 16 Software Peace of Mind in Action
???
Data file: Store test case information
Test Automation Implementation

9. 4/27/2015
9
Slide 17 Software Peace of Mind in Action
???
Object map: Store object information
Test Automation Implementation
Slide 18 Software Peace of Mind in Action
???
Engine components: Interpret test case information to
dynamically create “automation script” during execution
Test Automation Implementation

10. 4/27/2015
10
Slide 19 Software Peace of Mind in Action
???
Logging file: The logging file will capture all the events
that take place in the Interpret Engine. This will help in
debugging the Interpret Engine
Reporting engine: The reporting will be created in html
format. The reporting can be set in two modes; info and
debug. In info mode, the html report will only report on
Verify Action. In debug mode, the html report will
contain all steps in the test case
Test Automation Implementation
Slide 20 Software Peace of Mind in Action
???
Automated 830 test cases
Integration level test cases with average of 38 steps/test
case
20 test cases are used for smoke testing by pairing with
continuous integration tool
Test cases are divided into three priorities (high, medium,
low)
Development and QA determine tests to run
Project Outcome

11. 4/27/2015
11
Slide 21 Software Peace of Mind in Action
???
Execution of full automation suite takes around 3
hours using Sauce Lab
Manual testing would have taken 4 resources 40 hours to
execute the same suite
Maintenance of the suite, on average, is less than 8 hours
per execution
New enhancements for the automation framework
Data file will be replaced by database tables
A web UI will be created to make test case creation faster
A test execution scheduler will be created to manage off-
hour execution
Outcome of the project
Slide 22 Software Peace of Mind in Action
???
There are many benefits to leverage Selenium for test
automation. However, it is important to create a
detailed plan and roadmap to ensure success.
Conclusion