Seminar presentation from Tampere Technical University, testing day 2016: http://www.cs.tut.fi/tapahtumat/testaus16/ Open Source project: https://github.com/symbionext/DockerizedRobotFramework

1. Company Confidential
Company Confidential
Scalable Test Automation
TTY - Testauspäivä 2016
Sakari Hoisko

2. Company Confidential
WE ARE SYMBIO
Symbio is a global BPO and R&D technology development
company offering solutions and services for end-to-end customer
care. We help clients develop technologies and customer care
services that enable them to manage their business and
customer relationships more efficiently. Our vision is to become
the world’s leading provider of R&D Engineering and BPO
services. Together with our partners, we design, build,
implement and support innovative customer engagement
platforms by seamlessly integrating our robust customer
insights, technology and world class business services.
2

3. Company Confidential
WHY AUTOMATION?
Helping you focus what is most important!
3

4. Company Confidential
4
TOOLS

5. Company Confidential
ROBOT FRAMEWORK
Compact sheet: https://public.centerdevice.de/0db44f0e-c485-41be-ba4a-85218b9b0542
› Test Automation Framework with html and
XML result output
› Include tagging and basic setup/teardown
tree functionality to suites and testcases
› Has a flexible keyword-driven approach to
describe and implement tests.
› Keywords can be stored in Resource-Files
to separate implementation details from
tests.
› Main programming languages for
implementing test functionality are Java
(JYBOT) and Python (PYBOT).
› There are ready-made test libraries
available for a lot of technologies.
› Implementing tests is supported by the
Robot Framework IDE(RIDE).

6. Company Confidential
DOCKER –> PACK –> SHIP –> RUN
Cheat sheet: https://github.com/wsargent/docker-cheat-sheet
› PACK software to docker container image
- Defined by using docker file whitch is pure text file
- Include just SW and it dependencies, nothing else.
- Built from docker file
- Minimize maintain of platform / security / updates
› SHIP software by using
- Container image docker file
- Container image by using private or public registry
› RUN docker container
- Isolated user space, shared kernel space
- Running application by single “docker run” command
from “docker image” template
- Is like lightweight “virtual machine” without OS and
HW emulation
- Include HW resources allocation, clustering, HA,
networks, port mapping, etc…

7. Company Confidential
ROBOT TEST SUITES / CASES DESIGN
› Unique independed test suites
- Test cases are executed parallel so there should not
be dependency between test cases which effects to
other testcase in any phase/step of test case
execution
- Test data should be unique
- All actions inside of test case should be unique (=if
same table is droped and added in different
testcases we have a problem).
› Timings
- All timings should be solved by loops / triggers /
polling (=not sleeps/etc ugly hacks)
- Is service up&running? Test it by using “run
keyword until” keyword functionality.
Keep parallel execution in mind while TA developing.

8. Company Confidential
PABOT VS (PY/JY)BOT
Details: https://github.com/mkorpela/pabot
› Parallel executor for Robot Framework test suites by using pybot /
jybot
› With Pabot you can split one execution into many and save test
execution time.
› Implemented as library to Robot Framework
› Speed up
- CI pipeline
- Test automation
- Developer feedback
› Require
- multiple robot host instances with same library/dependency contents
- Instances are defined to valueset.dat whitch is provided to pabot call as
argument
- pabot.PabotLib to test suite(s)/test case(s) (=provides keywords that
will help communication and data sharing between the executor
processes).
› Include locks that could be used to sync test suites/cases between
each other inside of pabot cluster
VS.
J
Y
B
O
T
P
Y
B
O
T
PABOT

9. Company Confidential
9
TOOLS TOGETHER => SCALABLE TEST
AUTOMATION

10. Company Confidential
ROBOT FRAMEWORK IN DOCKER CONTAINER
Same container everywhere.
› TA container image include:
- Pybot, Jybot and Pabot
- Required Robot Framework Libraries
- Tools (=browsers, RIDE, firebot, Zap, etc…)
- SSH server with known account (FYI: pabot doesn’t support key
based auth)
- All of those dependencies with correct order and
required versions (=standalone installation means gray hairs to them
who made this without docker)
› Same version of different components are in use both in
developer machine as in pipeline (if test suite works in developer
machine then it works in CI TA)
› GUIs could be forwarded outside of container by using X11
forwarding (=cool selling demo to managers when selenium2library is used)
TA
Cluster
TA development

11. Company Confidential
11
EXAMPLE FROM DEMO –
TEST AUTOMATION FRAMEWORK DOCKER FILE

12. Company Confidential
ROBOT FW CONTAINER CLUSTER
Same container everywhere.
› Master container
- Started by CI / developer
- Shut down by CI / developer
- All phases are executed in here
- All test results are collected in here by pabot
› Phase 1:
- Input: folder where suites located
- Calculate amount of test suites
- Start own TA container for every test suite (=start TA cluster containers)
- Generate parameter file where TA cluster is defined
- Build up platform to sandbox
- Generate parameter file where platform interfaces are defined
› Phase 2:
- Input: cluster and platform definition file
- Start PABOT (test suites) execution by parallel against platform sandbox
› Phase 3:
- Input: cluster and platform definition file
- Teardown shutdown platform sandbox and TA cluster
TA cluster containers

13. Company Confidential
POOR MAN’S PERFORMANCE TEST
Could be part of CI testing pipe.
› Start also system monitoring in phase 1.
› Pick up dedicated test case(s) that loads
platform system as widely as you can.
- Log in is usually good start some load authentication, DB,
frontend, backend, etc…
› Create test case(s) to keyword resource file
› Call same keyword(s) with uniq test data
from multiple test suites parallel by using
pabot
› Monitor platform simultaneously when test
set is under execution
› Stop system monitorin in phase 3.

14. Company Confidential
14
DEMO TIME - SCALABLE TEST AUTOMATION

15. Company Confidential
DOCKERIZING A ROBOT FRAMEWORK
To get official robot framework docker image to https://hub.docker.com/explore/
› Community based Open Source development
› Figuring out:
- What is base content?
• Execution tools?
• Libraries?
• R&D tools?
• Example test suites/cases (=Demo)?
- Reusable in different use cases?
- How to get content to be more configurable?
- Split configuration between:
• build time configuration (inside of image)
• run time configuration (deploy container from image)
› Contribute your view/points:
https://github.com/symbionext/DockerizedRobotFramework

16. Company Confidential
16
FRIENDLY ASK QUESTIONS?

17. Company Confidential
FINAL TIP => CD-PIPELINE WITH DOCKER
Jenkins integration - Poor man’s perf behind of jenkins in Continuous Delivery pipeline
- Jenkins could start docker containers as slaves
dynamically
• With docker plugin: https://wiki.jenkins-
ci.org/display/JENKINS/Docker+Plugin
• Master container could be integrated to
Jenkins as dynamic slave
- Jenkins could be installed to system by using
docker container
• Jenkins official docker container:
https://hub.docker.com/_/jenkins/
What else you need to build up
test automation to your system?
Jenkins Slave
Container

18. Company Confidential
CONTACT INFO
http://www.symbio.com/contact-us/

19. Company Confidential
19
THANKS.