Presentation by Monika Thül (Björnsen Beratende Ingenieure GmbH) at the Delft-FEWS International User Days 2018, during the Delft Software Days - Edition 2018. 7 & 8 November 2018, Delft.

1. Setting up a test environment in the context of FEWS
07/11/2018
Setting up an automated test environment in the
context of Delft-FEWS (using 'workflowTestRun')
07/11/2018 Gernot Belger, Monika Thül Dr. Bastian Klein, Dennis Meißner
Delft-FEWS - International User Days, 07. - 08. November 2018, Delft

2. Setting up a test environment in the context of FEWS
07/11/2018
What does „testing“ mean?
Agenda
Could testing be useful in the context of Delft-FEWS?
Onboard means of testing within Delft-FEWS
Extended test environment
Summary

3. Setting up a test environment in the context of FEWS
07/11/2018
The aim:
Ensure that executing a function in a given, known initial situation
always yields the expected result.
What does „testing“ mean?

4. Setting up a test environment in the context of FEWS
07/11/2018
Could testing be useful in the context of FEWS?
▪ Active / intentional changes to the configuration
▪ Setup of a ModuleInstance / Workflow
▪ Side effects from other changes
▪ Unintentional changes to the system
▪ Update of operating system
▪ Update of modelling software
▪ Delft-FEWS Update / Upgrade

5. Setting up a test environment in the context of FEWS
07/11/2018
WorkflowTestRun (XML configuration overview)
General definitions section
• List of directory definitions for later use in activities
• Set system time
List of activities
• Purge: purge a single file or set of files within a directory
• Copy: copy single file to given location
• Workflow
• Export time series (PI time series)
• Export logs: export log messages produced during last workflow run
• Set system time: set new system time
• Compare: compares two files (export files)
• Refresh: refresh datastore
• Contains: search text in text file
• Sleep
Onboard means of testing within Delft-FEWS

6. Setting up a test environment in the context of FEWS
07/11/2018
WorkflowTestRun, started within FEWS: <F12>
Onboard means of testing within Delft-FEWS
+ easy to use
execution information is presented in the logging area & FEWS log
+ familiar way
- log can‘t be analysed at a glance
- works on the current localDatastore
- results may interfere with current results
- each WorkflowTestRun has to be called seperately

7. Setting up a test environment in the context of FEWS
07/11/2018
WorkflowTestRun, start via command line („from bin dir“)
• Needs a dedicated FEWS to be set up, main difference in corresponding INI file
main.class=nl.wldelft.fews.system.workflowtestrun.WorkflowTestRun
Onboard means of testing within Delft-FEWS
+ setting up a parallel data structure can reduce interference with working FEWS
+ WorkFlowTestRuns can be started via BAT files
+ additional preliminary work can be done (e.g. providing a localDatastore)
because FEWS isn‘t running
- BAT programming is rather for specialists,
adding tests might be difficult for normal users
- LOG files have to be analysed by the user
- test results still have to be prepared in advance

8. Setting up a test environment in the context of FEWS
07/11/2018
Wish list
• Easy to use, results easy to analyse
• Executes several WorkflowTestRuns
• Simple configuration
• Simplifies set up a WorkflowTestRun
• Assistance for providing test data (localDatastore, results)
Extended test environment

9. Setting up a test environment in the context of FEWS
07/11/2018
General workflow
• Start logging
• Read configuration
• Safeguard FEWS directories (localDatastore, import, export, report,...)
• Loop through list of defined tests
• Prepare initial situation (provide localDatastore, import data,...)
• Execute WorkflowTestRun and retain log files
• Check results (contents or at least existence)
• Clean up
• Restore safeguarded directories
Additional mode to assist in setting up data
Extended test environment

10. Setting up a test environment in the context of FEWS
07/11/2018
WorkflowTestRun (reduced set of necessary elements)
General definitions section
• Reduced list of directory definitions for later use in activities
• Set system time
List of activities
• Purge: purge a single file or set of files within a directory
• Copy: copy single file to given location (only for special result files)
• Workflow
• Export time series (PI time series)
• Export logs: export log messages produced during last workflow run
• Set system time: set new system time
• Compare: compares two files (export files)
• Refresh: refresh datastore
• Contains: search text in text file (only for special log files)
• Sleep
Extended test environment

11. Setting up a test environment in the context of FEWS
07/11/2018
Encouragement: proceed step by step
• Set up an environment that suits your needs
• Agree on general structure of tests
• Arrange for the tests to be regularly carried out
• Implement or work on a test as you create or work on a workflow
Provide quality for
• Your organisation‘s system
• Your customer‘s system
Demand quality from
• Your service provider
Summary

12. Setting up a test environment in the context of FEWS
07/11/2018
Thank you for your attention
07/11/2018 Gernot Belger, Monika Thül Dr. Bastian Klein, Dennis Meißner