Solving the Automation Puzzle
@bendet_ori

About me - @bendet_ori
5.5y in HPE Software in various managerial QA roles
Michael’s father | Naomi’s husband
For small talk later:
Today – Inbound PM for UFT & a new cloud
offering
Largest Pharm.
Customer in
Europe
Critical Defect in
Diaper

State of Automation – the poll!
3

4

5

6

7

To Business!
8

Assumption #1
9
Vendors only use their own tools

Assumption #2
10
Evil corporates hate Open Source

Assumption #3
11
Everybody is doing automated
testing

My Automation Journey
12

Project #1
AUT: Analytic Platform
Automation: Internal Standalone Tool
13

Analytic Platform for IT Executives (v 1.0)
– AUT technology Stack
– Glassfish Server
– Flex + GWT
– SAP BODS for ETLs
– SAP BOE for BI
– MSSQL
– Automation
– People doing automation: 0.5 / 5
– Focusing mainly on APIs
– Java Beans (EJBs)
– Internal tool called FIST
– ROI: LOL
14
3.5 hours to install

15

16

17

18
An extendable Java class

19

Why?
20

Lessons Learned
Standalone Tool developed internally
– Cons
– Nobody knew about the tool, wasn’t cool, no buzz around it
– External tool: unable to get DEV to cooperate with automation (or even install the tool)
– Manually triggered (no part of the CI process)
– Almost every new test required changes to be done by DEV (expose new API methods)
– No direct access to source code
– Pros
– Small investment
– QTTV
– Easily extendable
– Stability
21

Analytic Platform for IT Executives (v 1.0 2.0)
– AUT technology Stack
– Glassfish Server
– Flex + GWT
– SAP BODS for ETLs
– SAP BOE for BI
– MSSQL
– Automation
– People doing automation: 4 / 10, dedicated developer to assist
– Focusing mainly on APIs Automation installation, E2E flows
– Java Beans (EJBs) REST APIs
– Internal tool called FIST Internally built framework REST client, Selenium, Flex Monkey
– Automatic Deployment Solution: internally developed tool called Slick
– ROI: 3 MD each sprint  1 MM per release
22
Automatic Deployment Solution

23
Saved over 10K MH

Why?
– High demand for automation coverage
– Developers selected the automation framework
– Wanted to work together with developers
– Invested in automated installation as well
24

Lessons Learned
Internally built Automation Framework (inside the IDE)
– Cons
– Required more coding skills
– Large effort of getting things started
– Harder to be used by less-technical testers
– UI automation stability
– Pros
– Harness developers into the automation
– Developers re-used testing assets for their own benefits
– Testers have access to source code
– Part of the CI process
25

Project #2
AUT: Performance Testing tool
Automation: Open Source
26

Performance Testing tool
– AUT technology Stack
– nodeJS
– AngularJS
– Internal SQLlite
– Automation
– People doing automation: 1 / 4
– System tests leveraging developers assets
– Focus on API testing, sanity level UI testing
– Application Modeling for less-technical testers
– Protractor, Mocha, Jasmine
– ROI: 1.5 MD / sprint
27
Protractor

Without the framework
With the framework
REST API

Without the framework
With the framework
UI:

Test (created by non-technical engineer)

Why?
– Best available choice for the technology stack
– 1 automation engineer working on the framework, others re-using the assets
31

Lessons Learned
Leveraging DEV assets and extending into our own framework (+ modeling)
– Cons
– Modeling takes time
– Technical engineer becomes the bottleneck
– UI automation stability
– Pros
– Use existing developers assets
– Extend automation coverage using non-technical engineers
– Modeling eases tests maintenance
32

Project #3
AUT: Firefox plugin
Automation: Commercial Tool (LeanFT)
33

TruClient?
TruClient is a tool for recording Web-based applications. It is used inside LoadRunner for
performance testing on the browser level
34
1. TruClient Sidebar
2. TruClient Toolbox
3. Firefox browser
4. Application Browser
Window
5. TruClient Sidebar
Status Pane

TruClient
– AUT technology Stack
– Firefox Plugin
– Pure Web
– WPF
– Windows app
– Automation
– People doing automation: 2 / 6
– Did not have an automation suite as they could not find a tool which has automation abilities for the full flow –
including the three technologies.
– Supports the three main browsers and needs an automation tool that can identify and test the objects in all the
supported browsers
– Selected LeanFT as the tool
35

The Automation Suite
36
LeanFT Application
Model
Containing the full AUT
hierarchy, the App
Model displays a
Modular view of all the
objects implemented in
the tested application

The Automation Suite
37
LeanFT test
- Everything is written in the IDE, Dev have
access to tests (and used them for sanity)
- Test code is completely reusable among the
whole test suite
- The test is authored once, and can be run on
all three browsers
- In the test setup, the TruClient launcher –
written in WPF - is started and being used,
and during the test, web and standard
windows technologies are tested.

Why?
– Cross Technology support (Desktop & Web)
– Script once – run of all browsers
– Re-use, share testing assets with Dev
38

Want to buy a new automation framework for 1 shekel??
39

The Automation Council
40

41

The Guidelines
42

Rule #1
43
Accessible in the developers
workspace

Rule #2
44
Cross Browsers/Technology
Support

Rule #3
45
Ability to easily model the UI

Rule #4
46
Full support for REST APIs testing

Rule #5
47
DB Layer

Rule #6
48
Ability to combine UI/API/DB in 1
flow

Rule #7
49
Messaging Parser (json, XML, etc.)

Rule #8
50
Parameterization of tests

Rule #9
51
CI/CD complaint

Rule #10
52
Modularity to allow re-use by less
tech eng

53

FIST Selenium built FWK LeanFT UFT
IDE X √ √ X
Cross
Browsers/Technology
X Mobile/Web √ √√√
Model the UI X X* √ √
REST API testing √ X* X* √
DB Layer √ X* X* √
Combine UI/API/DB X X* X* √
Parsers √ X* X* √
Parameterization √ √ √ √
CI/CD X √ √ √
Modularity for less-tech X X √ √
Cross Platforms X √ X X
Lightweight X √ √ X
54= can be added by user

Summary
55

Assumption #1
56
Vendors only use their own tools

Assumption #1
57
Vendors only use their own tools
It’s not about the tool

Assumption #2
58
Evil corporates hate Open Source

Assumption #2
59
Evil corporates hate Open Source
We love Open Source!!

Assumption #3
60
Everybody is doing automated
testing

Assumption #3
61
Everybody is doing automated
testing
Everybody is using automated
testing

Take Home Message
62

“It’s never about the tool,
it’s about finding the right tool
for the right project”
63

Thank You!
Contact me: @bendet_ori
r2d2@hpe.com
64