Containerized end-to-end testing provides advantages like isolation of environments, reproducible application environments, and optimized parallel execution. Sakuli is an open source tool that allows defining end-to-end tests using JavaScript or Java and executing them within Docker containers. It supports testing graphical user interfaces using technologies like X11 forwarding or VNC. Sakuli provides functions for web testing and interacting with applications and defines a structure for test cases with initialization, test steps, and exception handling. It can integrate with monitoring and CI tools to report test results.
2. Characteristics of End-2-End Testing
Different types of testing:
Regression tests
Functional approval tests
Parallel tests with GUIs are complex
Stateful tests: user logins, sessions, history
Setup and cleanup of test data
Manual effort > effort for test automation
3. Advantages of Container Technology
Isolation of environments
Repository for versioning and distribution
Reproducible application environment
Dockerfile, docker-compose.yml
Optimized for parallel execution and cloud systems
Less memory and CPU overhead (shared Linux kernel)
Starting containers on-the-fly
Unique command line interface (orchestration tools)
4. Containerized GUIs
### start the docker container via x-forwarding
docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:rw rasch/inkscape
### start the docker container with VNC interface
# connect via URL: http://localhost:6911/vnc_auto.html?password=vncpassword
docker run -it -p 5911:5901 -p 6911:6901 consol/ubuntu-xfce-vnc
docker run -it -p 5912:5901 -p 6912:6901 consol/centos-xfce-vnc
docker run -it -p 5913:5901 -p 6913:6901 consol/ubuntu-icewm-vnc:dev
docker run -it -p 5913:5901 -p 6914:6901 consol/centos-icewm-vnc:dev
5. What's provided by ?
Category
Web tests through HTML selectors
Restricted to browser content
Open Source & Java API
Headless execution
Test writing assistance
(recorder, tag finder)
Automatable / Test result reporting
(CI, DB, monitoring environment)
8. Test Case Structure
// testcase.js
/*************************************
* Initialization
*************************************/
_dynamicInclude($includeFolder);
var testCase = new TestCase( 60, 70);
var env = new Environment();
var appNotepad = new Application( "gedit");
var region = new Region();
/******************************
* Description of the test case
******************************/
try {
//...
/************************************************
* Exception handling and shutdown of test case
**********************************************/
} catch (e) {
testCase.handleException(e);
} finally {
testCase.saveResult();
}
10. Fluent API for UI Testing
// testacase.js
/*** open calculator app ***/
var calculatorApp = new Application( "galculator" ).open();
testCase.endOfStep( "Open Calculator" , 3);
/*** calculate 525 + 100 ***/
var calculatorRegion = calculatorApp.getRegion();
calculatorRegion.type( "525")
.find( "plus.png" )
.click()
.type( "100");
calcRegion.find( "result.png" ).click();
calcRegion.waitForImage( "625", 5);
testCase.endOfStep( "calculate 525 +100" , 20);
11. Custom Functions
// e.g. excluded into a separate common.js
/**********
* Combine click and highlight
*********/
function clickHighlight ($selector) {
_highlight($selector);
_click($selector);
}
/***************
* Open PDF in native viewer
**************/
var PDF_EDITOR_NAME = "masterpdfeditor3" ;
function openPdfFile (pdfFileLocation ) {
return new Application(PDF_EDITOR_NAME + ' "' + pdfFileLocation + '"').open();
}
12. Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
private static final String CITRUS_URL = "http://www.citrusframework.org/" ;
private Environment env;
private Region screen;
@BeforeClass
@Override
public void initTC() throws Exception {
super.initTC();
env = new Environment();
screen = new Region();
browser.open();
}
@Override
protected TestCaseInitParameter getTestCaseInitParameter () throws Exception {
return new TestCaseInitParameter( "test_citrus" );
}
@Test
public void testMyApp() throws Exception {
// ... testcode
}
}
For the Maven dependencies, take a look at the .Java DSL Documentaion
18. Bakery Demo
git clone https://github.com/toschneck/sakuli-example-bakery-testing.git
# start jenkins
jenkins/deploy_jenkins.sh
# start OMD montioring
omd-nagios/deploy_omd.sh
# start the build of the application images
bakery-app/app-deployment-docker-compose/deploy_app.sh
#start tests
sakuli-tests/execute_all.sh
#start tests for monitoring
sakuli-tests/execute_all_4_monitoring.sh
19. What's next?
Headless execution - Linux: VNC & Docker Windows: ?
Cloud-ready container forOpenShift (security, source-2-image)
Video recording of the test execution (error documentation)
Web UI to handle Sakuli test suites
Connect 3rd-party test management tools(HP QC, TestRail, ...)
Improve test result presentation in CI tools
Selenium as an alternative to Sahi
Implement Junit 5 test runner
