In this talk, I present an overview of three open source tools for enabling research and development of scientific workflow systems and applications: - SimGrid: https://simgrid.org - WRENCH: https://wrench-project.org - WfCommons: https://wfcommons.org

1. Modeling and Simulation of Parallel and
Distributed Computing Systems with
SimGrid, WRENCH, and WfCommons
Rafael Ferreira da Silva, Ph.D.
https://rafaelsilva.com

2. https://wrench-project.org https://wfcommons.org https://simgrid.org
disconnect between
CI Theory and Practice
2
Many theoretical
results are not useful
to practitioners
One well-known reason is
that theoretical results are
obtained with models that,
to be tractable, are often
unrealistic or unattainable
in practice
As a result,
practical work must
be experimental

3. https://wrench-project.org https://wfcommons.org https://simgrid.org
real-world
Experiments are Limited
3
One is limited to particular platform
configurations (and sub-configurations)
How can “what if?” scenarios be explored?
How can generality be claimed?
One is limited by specifics of the software
infrastructure that impose constraints on CI
application executions
Modifying complex software stacks (often written
by others) just to test out ideas is not feasible
Limited Experimental
Scope impedes
progress / discovery

4. https://wrench-project.org https://wfcommons.org https://simgrid.org
resorting to
Simulation
4
When one works in an experimental
field in which experiments are
problematic, one resorts to simulation
In some fields of Computer Science
simulation is a standard research and
development methodology
So, what about
using simulation to
drive CI Research &
Development?

5. https://wrench-project.org https://wfcommons.org https://simgrid.org
Simulation-driven
engineering life cycle
5
Research
Idea
Design of
Research
Solution
Design of
Workflow
Simulator
Accurate
Workflow
Simulator
Experimental
Simulation
Evaluation of
Simulation
Results
Research
Product
Implementation
onto Workflow
System
unsatisfactory results
The ability to easily develop
accurate CI simulators, from
which research products
evaluated via experimental
simulation could be
seamlessly integrated into
actual CI platforms
R. Ferreira da Silva, H. Casanova, R. Tanaka, F. Suter (2019). Bridging Concepts and
Practice in eScience via Simulation-driven Engineering. Workshop on Bridging from
Concepts to Data and Computation for EScience (BC2DC’19), 15th International
Conference on EScience (EScience). DOI: 10.1109/eScience.2019.00084

6. https://wrench-project.org https://wfcommons.org https://simgrid.org
The SimGrid
simulation framework
6
H. Casanova, A. Giersch, A. Legrand, M. Quinson, F. Suter. Versatile,
Scalable, and Accurate Simulation of Distributed Applications and
Platforms. Journal of Parallel and Distributed Computing, Elsevier,
2014, 74 (10), pp.2899-2917. http://hal.inria.fr/hal-01017319
SimGrid is a mature research project
Provides simulation models of hardware/software stacks
that are accurate (validated/invalidated) and scalable (low
computational complexity, low memory footprint)
SimGrid is open source usable software
Vibrant developer and user community, continuous
integration testing, one release every 6 months
SimGrid is versatile
Used for (combinations of) Grid, HPC, Peer-to-Peer,
Cloud, and Fog simulation projects
https://simgrid.org
First developed in 2000
latest release: v3.28 (July 2021)

7. https://wrench-project.org https://wfcommons.org https://simgrid.org
SimGrid’s
Philosophy
7
SimGrid provides
low-level abstractions
you can do anything with it
implementing a simulation of a complex system
is cumbersome
Critical analysis: In [Kecskemeti et al.’14] pinpoints exactly the above trade-off:
Summary: SimGrid is more scalable and validated than competing frameworks,
but just too much work when wanting to simulate a complex CI system

8. https://wrench-project.org https://wfcommons.org https://simgrid.org
SimGrid’s
Overview
8
https://simgrid.org/doc/latest/Introduction.html

9. https://wrench-project.org https://wfcommons.org https://simgrid.org
SimGrid’s
Key Concepts
9
S4U interface
(SimGrid for you)
Actor
Engine
Elements
Mailbox
Active entities executing an
application
Simulation engine (singleton)
Communication
rendezvous, with which
actors meet each other
Disk
Host
Resources
Link
Resource on which actors
can write and read data
Actor location, providing
computational power
Interconnecting hosts
https://simgrid.org/doc/latest/app_s4u.html

10. https://wrench-project.org https://wfcommons.org https://simgrid.org
The WRENCH
simulation framework
10
H. Casanova, R. Ferreira da Silva, R. Tanaka, S. Pandey, G. Jethwani, W. Koch, S.
Albrecht, J. Oeth, and F. Suter, "Developing Accurate and Scalable Simulators of
Production Workflow Management Systems with WRENCH", Future Generation
Computer Systems, vol. 112, 2020. DOI: 10.1016/j.future.2020.05.030
https://wrench-project.org
First developed in 2017
latest release: v1.9 (August 2021)
simulation building blocks
Reuse of existing and implementation of
novel simulated core CI services
simulation integration
Development of software elements that
make it possible to augment existing
systems with simulation capabilities in a
language-agnostic manner
Workflow Systems
simulation accuracy
Reliance on existing and development of
novel accurate and validated simulation
models
simulation scalability
Ability to run large simulations quickly
on a single computer with low compute,
memory, and energy footprints
Developed software elements make it possible to augment CI systems with simulation
capabilities to transform the way in which these systems are experimentally evaluated
The above will be demonstrated for several production Workflow Systems that support
critical scientific applications

11. https://wrench-project.org https://wfcommons.org https://simgrid.org
11
WRENCH’s
Simulated Core CI Services
Compute Services
Provide mechanisms for
executing application tasks,
which entail I/O and
computation
cloud
bare-metal virtualized cluster
batch-scheduled cluster
Storage Services
Store application files,
reading/writing by the
compute services
File Registry Services
Databases of key-value pairs of
storage services and files
replicas
Network Proximity Services
Monitor the network and
provide host-to-host network
distances
Workflow Systems
Mechanisms for executing
workflow applications
disk object storage
I/O cache burst buffer
scheduling
energy
resource provisioning
Simulation core
All necessary simulation models and base abstractions (computing,
communicating, storing), provided by SimGrid

12. https://wrench-project.org https://wfcommons.org https://simgrid.org
12
WRENCH’s
Simulation Accuracy and Scalability
Accuracy
The ability to capture the behavior of a real-world
system with as little bias as possible
Scalability
The ability to simulate large systems with as
few CPU cycles and bytes of RAM as possible

13. https://wrench-project.org https://wfcommons.org https://simgrid.org
The WfCommons
workflow research and development framework
13
T. Coleman, H. Casanova, L. Pottier, M. Kaushik, E. Deelman, and R. Ferreira da Silva,
"WfCommons: A Framework for Enabling Scientific Workflow Research and Development,"
arXiv preprint arXiv:2105.14352, 2021.
https://wfcommons.org
First developed in 2020
latest release: v0.7 (August 2021)
A traditional approach for testing,
evaluating, and evolving workflow
systems is to use full-fledged software
stacks to execute applications on
distributed platforms and testbeds
WfCommons is a framework that provides
a collection of tools for analyzing
workflow execution traces, producing
realistic synthetic workflow traces, and
simulating workflow executions

14. https://wrench-project.org https://wfcommons.org https://simgrid.org
14
WfCommons’
Concept
WfFormat
WfInstances
WfChef
WfGen
WfSim
Catalogs of workflow instances
Common JSON workflow instance format
Automates the construction of synthetic workflow generators
Generation of realistic synthetic workflow instances
Catalogs of workflow simulators supporting WfFormat

15. https://wrench-project.org https://wfcommons.org https://simgrid.org
SimGrid, WRENCH, and WfCommons
in Numbers
15
since 2020
since 2018
since 2001
165 workflow
execution instances
9 workflow applications
3 workflow simulators
4 WfCommons-enabled
research publications
6 stable
software releases
13 stable
software releases
30+ stable
software releases
18 WRENCH-enabled
research publications
540+ SimGrid-enabled
research publications
6 open source software
leveraging SimGrid
13 simulators for
pedagogic modules
40+ contributors for
SimGrid’s core software
16 contributors for
WRENCH’s core software

16. Modeling and Simulation of Parallel and
Distributed Computing Systems with
SimGrid, WRENCH, and WfCommons
Rafael Ferreira da Silva, Ph.D.
https://rafaelsilva.com
8/27/21
https://wrench-project.org https://wfcommons.org https://simgrid.org
Thank You