Bei der Konzeption von End-­2-­End­-Tests ist eines der größten Probleme die Frage, wie die Testausführung möglichst robust, reproduzierbar und skalierbar gestaltet werden kann. Diese Hürde lässt sich mit klassischen Ansätzen nicht überwinden. Einen eleganten Ausweg bieten in Container verpackte Testumgebungen. Durch sie wird es möglich, einen definierten Systemstand reproduzierbar aufzurufen und Tests performant auszuführen. Im Vortrag wird gezeigt, wie z.B. parallele GUI­-Tests in verschiedenen Umgebungen zur Qualitätssicherung beitragen. Die Beispiele sind mit den Open-­Source­-Tools Sakuli und Docker realisiert und testen Web­- und Rich-­Client­-Applikationen. Lernziele: Ziel ist es, dem Zuhörer aufzuzeigen, wie das Potenzial von Container­-Technologien genutzt werden kann, um die Softwarequalität zu erhöhen und den manuellen Testaufwand drastisch zu verringern. Eine abschließende Bewertung der Erfahrungen sowie ein Ausblick auf weitere Einsatzszenarien und Entwicklungsschritte runden den Vortrag

1. Containerized End­2­End Testing
+
,
Tobias Schneck ConSol Software GmbH

2. Test Pyramid
Robust
Independent
Isolated
Fast

3. Characteristics of End­2­End Testing
Different types of testing:
Regression tests
Functional approval tests
Parallel tests with GUIs are complex
Stateful tests: user logins, sessions, history
Setup and cleanup of test data
Manual effort > effort for test automation

4. Advantages of Container Technology
Isolation of environments
Repository for versioning and distribution
Reproducible application environment
Dockerfile, docker­compose.yml
Starting containers on­the­fly
Optimized for parallel execution
Less memory overhead
Shared Linux kernel

5. Containerized GUIs
### start the docker container via x­forwarding
docker run ­it ­e DISPLAY=$DISPLAY ­v /tmp/.X11­unix:/tmp/.X11­unix:rw rasch/inkscape
### start the docker container with VNC interface
# connect via URL: http://localhost:6911/vnc_auto.html?password=vncpassword
docker run ­it ­p 5911:5901 ­p 6911:6901 consol/ubuntu­xfce­vnc
docker run ­it ­p 5912:5901 ­p 6912:6901 consol/centos­xfce­vnc

6. What's provided by ?
Category
Web tests through HTML selectors
Restricted to browser content
Open Source & Java API
Headless execution
Test writing assistance
(recorder, tag finder)
Automatable / Test result reporting
(CI, DB, monitoring environment)

7. Nagios OMD Incinga
Monitoring Integration

8. Test Definition (Java Script)

9. Test Case Structure
// testcase.js
/*************************************
* Initialization
*************************************/
_dynamicInclude($includeFolder);
var testCase = new TestCase(60, 70);
var env = new Environment();
var appNotepad = new Application("gedit");
var region = new Region();
/******************************
* Description of the test case
******************************/
try {
//...
/************************************************
* Exception handling and shutdown of test case
**********************************************/
} catch (e) {
testCase.handleException(e);
} finally {
testCase.saveResult();
}

10. Call all Sahi Functions for Web Testing
// testacase.js
/************************
* Call Sahi Functions
***********************/
_navigateTo("http://sahi.example.com/_s_/dyn/Driver_initialized");
_highlight(_link("SSL Manager"));
_isVisible(_link("SSL Manager"));
_highlight(_link("Logs"))
_click(_link("Logs"))
testCase.endOfStep("Test Sahi landing page", 5);

11. Fluent API for UI Testing
// testacase.js
/*** open calculator app ***/
var calculatorApp = new Application("galculator").open();
testCase.endOfStep("Open Calculator", 3);
/*** calculate 525 + 100 ***/
var calculatorRegion = calculatorApp.getRegion();
calculatorRegion.type("525")
.find("plus.png")
.click()
.type("100");
calcRegion.find("result.png").click();
calcRegion.waitForImage("625", 5);
testCase.endOfStep("calculate 525 +100", 20);

12. Custom Functions
// e.g. excluded into a separate common.js
/**********
* Combine click and highlight
*********/
function clickHighlight($selector) {
_highlight($selector);
_click($selector);
}
/***************
* Open PDF in native viewer
**************/
var PDF_EDITOR_NAME = "masterpdfeditor3";
function openPdfFile(pdfFileLocation) {
return new Application(PDF_EDITOR_NAME + ' "' + pdfFileLocation + '"').open();
}

13. Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
private static final String CITRUS_URL = "http://www.citrusframework.org/";
private Environment env;
private Region screen;
@BeforeClass
@Override
public void initTC() throws Exception {
super.initTC();
env = new Environment();
screen = new Region();
browser.open();
}
@Override
protected TestCaseInitParameter getTestCaseInitParameter() throws Exception {
return new TestCaseInitParameter("test_citrus");
}
@Test
public void testMyApp() throws Exception {
// ... testcode
}
}
For the Maven dependencies, take a look at the .Java DSL Documentaion

14. Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
// ... initializing code
@Test
public void testCitrusHtmlContent() throws Exception {
browser.navigateTo(CITRUS_URL);
ElementStub heading1 = browser.paragraph("Citrus Integration Testing");
heading1.highlight();
assertTrue(heading1.isVisible());
ElementStub download = browser.link("/Download v.*/");
download.highlight();
assertTrue(download.isVisible());
download.click();
ElementStub downloadLink = browser.cell("2.6.1");
downloadLink.highlight();
assertTrue(downloadLink.isVisible());
}
@Test
public void testCitrusPictures() throws Exception {
browser.navigateTo(CITRUS_URL);
env.setSimilarity(0.8);
screen.find("citrus_logo.png").highlight();
env.type(Key.END);
screen.find("consol_logo.png").highlight();
}
}

15. Sakuli End­2­End Testing Container

16. Demo ­ Sakuli Container
# start the docker container
docker run ­it ­p 5911:5901 ­p 6911:6901 consol/sakuli­ubuntu­xfce
docker run ­it ­p 5912:5901 ­p 6912:6901 consol/sakuli­centos­xfce
docker run ­it ­p 5913:5901 ­p 6913:6901 consol/sakuli­ubuntu­xfce­java
docker run ­it ­p 5914:5901 ­p 6914:6901 consol/sakuli­centos­xfce­java
# start in parallel via docker­compose
# use docker­compos.yml from https://github.com/ConSol/sakuli/tree/master/docker
docker­compose up

17. Bakery Demo Setup

18. Bakery Demo Setup

19. Bakery Demo
git clone https://github.com/toschneck/sakuli­example­bakery­testing.git
# start jenkins
jenkins/deploy_jenkins.sh
# start OMD montioring
omd­nagios/deploy_omd.sh
# start the build of the application images
bakery­app/app­deployment­docker­compose/deploy_app.sh
#start tests
sakuli­tests/execute_all.sh
#start tests for monitoring
sakuli­tests/execute_all_4_monitoring.sh

20. What's next?
Headless execution ­ Linux: VNC & Docker Windows: ?
Cloud­ready container for OpenShift (security, source­2­image)
Video recording of the test execution (error documentation)
Web UI to handle Sakuli test suites
Connect 3rd­party test management tools (HP QC, TestRail, ...)
Improve test result presentation in CI tools
Selenium as an alternative to Sahi
Implement Junit 5 test runner

21. Links
ConSol/sakuli
Consol/sakuli­examples
toschneck/sakuli­example­bakery­testing
sakuli@consol.de

22. Thank you Mannheim!
Tobias Schneck
ConSol Software GmbH
Franziskanerstraße 38
D­81669 München
Tel: +49­89­45841­100
Fax: +49­89­45841­111
tobias.schneck@consol.
toschneck
info@consol.de
www.consol.de
ConSol