The focus of the presentation is to develop a framework and platform that supports the integration of multiple models, simulations, and data. My aim is to develop methods to integrate a set of simulated environments to make it possible to combine various independent simulators, developed by different domain experts. This would make it possible for researchers to build complex, multi-domain simulations by integrating existing and well-established simulators, so they can explore different alternatives and conduct low cost experiments.

1. Middleware Solutions for Integrated
Simulation Environments
Leila Jalali
http://www.ics.uci.edu/~ljalali/
ljalali@ics.uci.edu
Distributed Systems Middleware
11/28/2012 Leila Jalali Group

2. Outline
 Motivation for New Simulation Platforms
 Reflective Architecture and Methodology
 Synchronization in Multisimulations
 Modeling Synchronization Problem using concepts from Multidatabases
 A Hybrid Scheduling Approach
 Optimizations: Checkpointing and roll-back; Relaxation models
 An Emergency Response Case Study
 Data Exchange in Multisimulations
 Modeling Data Exchange using Transformation Graphs
 Data Transformation Plan Generation: Algorithms
 An Emergency Response Use Case
 Formal Specification of Multisimulations
 Modeling Multismulations in Maude for rapid prototyping and analysis
 Conclusions and Future work
11/28/2012 Leila Jalali
2 Distributed Systems Middleware
Group

3. Introduction
1
 Simulation: the process of designing a model
of a real world system and conducting
experiments with this model for our purpose: 2
cheaper, safer, easier, and quicker
 Planning and decision support:
 1. Defence simulations 3
 2. Emergency response simulations
 Domain specific testing and analysis: 4
 3. Traffic analysis
 4. Human behaviour study: crowd dynamics
5
 5. Network simulators
6
 Immersive synthetic platforms for training
 6. Fire fighter training simulators
11/28/2012 Leila Jalali
3 Distributed Systems Middleware
Group

4. Motivation for New Simulation
Platforms
 Many available simulators
 Operate on specific domains
 e.g fire simulators, transportation simulators
 Infeasible to build complex simulations entirely from
scratch
 Need domain expertise to have an in-depth understanding of
the phenomena
 Economic and organizational constraints
 The increasingly complex requirements
 Need ability to:
 Bring together simulators from various modeling domains:
Multisimulations
 Model and test larger and more complex scenarios
 Study cause- effect relationships to integrate simulators
11/28/2012 Leila Jalali
4 Distributed Systems Middleware
Group

5. Simulation Integration- historical
view
Distributed Interactive Simulation (DIS) High Level Architecture (HLA)
(1990–today) Army Projects (1996-today) Defense
1975 1980 1985 1990 1995 2000
SIMulator NETworking (SIMNET)
(1983–1990) Combat Simulators
Defense Community Aggregate Level Simulation Protocol (ALSP)
(1991–1997ish) War-gaming models
Dungeons and Dragons
Adventure Board Games Multi-User Dungeon (MUD)
(Xerox PARC) Games Multi-User Video Games
Internet & Gaming Community
11/28/2012 Leila Jalali
5 Distributed Systems Middleware
Group

6. Current approaches
DIS 1990 ALSP 1991 HLA 1996
 not fully  requires strict  requires the individual simulators to conform to
distributed adherence to the standard; might not be always possible [20]
 no time causality, i.e., all  The standard may not have been designed to
management simulation events handle the new simulator needs
service must be  Low level knowledge needed from the
 does not processed in practitioner [22]
manage time stamp order  Cost issues, Complexity [22]
latency and  proven to be  Current model registration in HLA needs a lot of
causality very system manual work [20]
 foundations specific and does  No support for semantic interoperability
for distributed not provide a  Transparency
real-time general Network- too big and mainly applied in defense
HLA is
simulation [6] interoperability [20]
HLA
MATLAB- HLA SLX Model-integration
2007 2000 2000 2011
solution [5] specific implementatio simulation generic modeling
to of a n designed language with
RTI environment tool,
modeling extensibility, uses a custom
communicati specifically for the
a development developed DSML
networks [27] integration into Matlab [28]
on for
based on HLA [29]
env. model definition
[30]
SESA 2007 Splash 2010 BPI 2003
Integration of multiple Health care domain, focuses on how data can be Business process integration
workflows transformed before it is used by another middleware to facilitate the integration
downstream component model of applications
11/28/2012 Leila Jalali
6 Distributed Systems Middleware
Group

7. General Challenges
 Managing Complexity of Interoperating Systems
 Analysis of cause- effect relationships
 Correctness of Multisimulations
 What, how, when to exchange the information
among multiple simulators?
 Time synchronization: causality correctness
 Data exchange: data transformations
 Scalability
11/28/2012 Leila Jalali
7 Distributed Systems Middleware
Group

8. General Challenges- Our Focus
 Managing Complexity of Interoperating Systems
 Analysis of cause- effect relationships  Methodology to develop
multisimulations using meta-models
and meta-level modules
 Correctness of Multisimulations
 Formal Specifications of
multisimulations
 What, how, when to exchange the information
among multiple simulators?  Modeling synchronization
problem using concepts from
multidatabases
 Time synchronization: causality correctness
 Hybrid approach for
 Data exchange: data transformations synchronization
 Relaxation models
 Using transformation
 Scalability graphs for data
exchange
11/28/2012 Leila Jalali
8 Distributed Systems Middleware
Group

9. Autonomous simulators with
specific platform issues
Simulator Platform
Simulator Platform simulator i simulator j
source code parameters databases source code parameters databases
11/28/2012 Leila Jalali
9 Distributed Systems Middleware
Group

10. Complex analysis
smart decisions using multisimulations
what-if analysis
policy making algorithms
Autonomous simulators with
specific platform issues
Simulator Platform
Simulator Platform simulator i simulator j
source code parameters databases source code parameters databases
11/28/2012 Leila Jalali
10 Distributed Systems Middleware
Group

11. Complex analysis
smart decisions using multisimulations
what-if analysis
policy making algorithms
run-time
RAISE Middleware
Time Synchronizer Data Transformers
Integrated simulation
environment
actions dependency actions
descriptors
type data data type
simulator i simulator j
specification Analyze specification
Observe and extract Reflect Autonomous simulators with
specific platform issues
Simulator Platform
Simulator Platform simulator i simulator j
source code parameters databases source code parameters databases
11/28/2012 Leila Jalali
11 Distributed Systems Middleware
Group

12. Reflective Architecture for Integrated
Simulation Environments- RAISE
Complex Applications
run-time
Data Exchange Time Synchronization
dependencies
Data Consistency
Synchronizer Controller
Transformers
Meta level
Lock-table
meta-actions
Lock Manager
External
Analyzer & Adaptor Data
Sources
Meta models
Structural specification: UML diagrams, metamodels
Interactions: dependency sets, interdependent data
Observe & Extract Reflect
Base level
INLET Drillsim Fire, Earthquake LTESim
(Transportation Model) (Activity Model) (Crisis Model) (Communication Model)
11/28/2012 Leila Jalali
12 Distributed Systems Middleware
Group

13. RAISE Methodology
Developer
(integrator)
Meta level
1. Reification: extract simulators’ meta-
data and construct meta-models
Base level
simulator1 simulator2 simulator3 Live data
preprocessing steps
Base level
Meta level
 UML diagrams, human in the loop
process
 Making the underlying simulators
more understandable
Back-up Slides
11/28/2012 Leila Jalali
13 Distributed Systems Middleware
Group

14. RAISE Methodology (cont.)
Developer
(integrator)
Meta level
Base level
simulator1 simulator2 simulator3 Live data
preprocessing steps
2. Analysis of meta-models:
extract inter-dependencies and
describe them using dependency
3. descriptors
Create wrappers for each
simulator to interface with meta-
level
11/28/2012 Leila Jalali
14 Distributed Systems Middleware
Group

15. RAISE Methodology (cont.)
Developer Decision-maker
(integrator) System-wide analyzer
run
multisimulation
Meta level
Run-time
meta-models
inter-dependencies
Base level
Base level
simulator1 simulator2 simulator3 Live data simulator1 simulator2 simulator3 Live data
preprocessing steps run-time execution
3. Run multisimulation:
2. Analysis of meta-models:  Wrappers decide whether to
extract inter-dependencies and communicate with meta-level or not
describe them using dependency  What, how, when to exchange the
3. descriptors
Create wrappers for each information among multiple
simulator to interface with meta- simulators
level  Ensuring correctness:
 time synchronization
15 Distributed Systems Middleware
11/28/2012 Leila Jalali  data management
Group

16. Outline
 Motivation for New Simulation Platforms
 Reflective Architecture and Methodology
 Synchronization in Multisimulations
 Modeling Synchronization Problem using concepts from Multidatabases
 A Hybrid Scheduling Approach
 Optimizations: Checkpointing and roll-back; Relaxation models
 An Emergency Response Case Study
 Data Exchange in Multisimulations
 Modeling Data Exchange using Transformation Graphs
 Data Transformation Plan Generation: Algorithms
 An Emergency Response Use Case
 Formal Specification of Multisimulations
 Modeling Multismulations in Maude for rapid prototyping and analysis
 Conclusions and Future work
11/28/2012 Leila Jalali
16 Distributed Systems Middleware
Group

17. The Synchronization Challenge
 Ensuring causal correctness in concurrently
executing simulators while preserving simulators’
autonomy
 An example of causality violations without an effective
time synchronization:
Fire
Simulator Fire & Smoke
Activity
Simulator Someone extinguished the fire
Visualization
time
Still the fire there? Conflict
11/28/2012 Leila Jalali
17 Distributed Systems Middleware
Group

18. Time Synchronization- related
work
Time In distributed In networks In distributed
Synchronizatio systems simulation
n
approaches
• Clock • Parallel Discrete
• Conservative • Cristian Synchronization
-Network time Event Simulation
• Optimistic algorithm protocol • Distributed
• Scaled real- • Berkley • Time scale Interactive
Transformation Simulation
time algorithm -Client-server
-Time stamp synch • Aggregated Level
• Lamport -Beacon node Simulation Protocol
broadcast
algorithm -Light weight time • High Level
synchronization Architecture
• Time Warp
11/28/2012 Leila Jalali • SPEEDS 18 Distributed Systems Middleware
Group

19. Time Synchronization Challenges
(1) Different simulators represent time differently and
implement varying time advancement mechanisms.
Time-stepped simulators Event-based simulators :
:
event 1 event 2 event 3 time
time
1: while (simulation still in progress) do
1: while (simulation still in progress) do
2: Event e= nextEvent;
2: for each tick do 3: while(e!=null) do
3: read data; 4: process(e);
4: modify data; 5: time= timestampe(e);
5: time = time + Δt ; 6: e= nextEvent;
6: end for 7: end while
7: end while 8: end while
(2) The source code of simulators is largely legacy.
 Need to change as little as possible in implementing time
19
synchronization
11/28/2012 Leila Jalali
Distributed Systems Middleware
Group

20. Meta-Synchronization Model
Multisimulation
dependencies meta-actions
Meta level
MetaSynchronizer
wrapper wrapper
wrapper wrapper
actions . . . actions
Base level
d . . . d’
Simulator i Simulator j
 Capture steps in a time-stepped simulator or events in an event-based
simulator in the concept of multisimulation action
 Synchronization is needed at the meta level to ensure causal
correctness (correctness criteria)
 Meta-level view:
 Delay: action is delayed
 Schedule: action is granted to be safely executed
[Inspired by work from multidatabase concurrency]
11/28/2012 Leila Jalali
20 Distributed Systems Middleware
Group

21. Dependencies

 Internal actions vs. External actions
 External actions access at least one data item which is an interdependent data
item
[Inspired by global/local transactions]
11/28/2012 Leila Jalali
21 Distributed Systems Middleware
Group

22. Dependency Preservation in
Schedules

11/28/2012 Leila Jalali
22 Distributed Systems Middleware
Group

23. Schedule Example

Formal Definition: back-up slides
11/28/2012 Leila Jalali
23 Distributed Systems Middleware
Group

24. Synchronization Protocol
Base-level Wrappers
Meta-level
 Wrappers are written
for each simulator to internal action allow
control the execution
of individual
simulators
11/28/2012 Leila Jalali
24 Distributed Systems Middleware
Group

25. Synchronization Protocol
Base-level Wrappers
Meta-level
 Wrappers are written
for each simulator to internal action allow
control the execution external action grant *
of individual external action allow
simulators
external action done
 Upon receiving
external actions: updates on interdependent data
schedule* at meta-
level
11/28/2012 Leila Jalali
25 Distributed Systems Middleware
Group

26. Synchronization Protocol
Base-level Wrappers
Meta-level
 Wrappers are written
for each simulator to internal action allow
control the execution external action grant *
of individual external action allow
simulators
external action done
 Upon receiving
external actions: updates on interdependent data
schedule* at meta-
level
meta-actions
internal action allow
external action grant
 Upon receiving updates external action allow
meta-actions are external action done
generated at meta-level updates
11/28/2012 Leila Jalali
26 Distributed Systems Middleware
Group

27. Solutions to Synch. problem

11/28/2012 Leila Jalali
27 Distributed Systems Middleware
Group

28. Lock Step Strategy
 Lock-step Scheduling: the most conservative
approach by having a lock step schedule
 All simulators advance step by step
 At any step they synchronize by locking at data item
level until the next step
 By locking interdependent data in the beginning of
action and releasing the locks at the end, we can
prevent deadlocks
 Our experiments show very high synchronization
time when using this approach
11/28/2012 Leila Jalali
28 Distributed Systems Middleware
Group

29. Conservative Strategy

current time=ti
...
a1 a2
next action of Sj is delayed:
...
a’1 a’2
current time=tj
11/28/2012 Leila Jalali
29 Distributed Systems Middleware
Group

30. Optimistic Strategy
 External actions from different simulations are allowed
whenever they are requested
 We accept the fact that dependency violations can
occur
 Resolve the violation when it does occur by aborting the actions that
caused the violation
current time=ti
...
a1 a2
next action of Sj is not delayed
a’1 a’2
current time=tj
11/28/2012 Leila Jalali
30 Distributed Systems Middleware
Group

31. Effectiveness of Scheduling Strategies
 Conservative and optimistic strategies may
become more (or less) effective as a
multisimulation progresses
 Initially, the cost of abort is small, so the optimistic
strategy performs better than conservative
strategy
 As the simulator proceeds, aborts costs become
increasingly high
 We propose a cost-driven hybrid approach that
combines the benefits of both the optimistic and
conservative strategies by considering the
underlying dependencies to make an informed
decision on whether to abort or delay an action Middleware
11/28/2012 Leila Jalali
31 Distributed Systems
Group

32. A Cost-Driven Hybrid Scheduling
Strategy
 Combines the benefits of both the conservative and
optimistic strategies
 Considers the expected cost of both conservative and optimistic
scheduling of each action, and chooses the method with the
lower expected cost
 Minimizes the expected execution time for each simulator in
the integrated execution
current time=ti
...
a1 a2 Need to make a decision to allow the next action of Sj or to de
a’1 a’2
current time=tj
11/28/2012 Leila Jalali
32 Distributed Systems Middleware
Group

33. How to make the decision?
duration of time that Sj may receive an update from Si that results in violation
current time=ti update on di
Si a1 a2
aborted
Sj a’1 a’k
… allowed not aborted
current time=tj next action= a’k
delayed necessary
unnecessary

11/28/2012 Leila Jalali
33 Distributed Systems Middleware
Group

34. How to make the decision?
duration of time that Sj may receive an update from Si that results in violation
current time=ti update on di
Si a1 a2
aborted
Sj a’1 a’k
… allowed not aborted
current time=tj next action= a’k
delayed necessary
unnecessary

11/28/2012 Leila Jalali
34 Distributed Systems Middleware
Group

35. Computing the costs

11/28/2012 Leila Jalali
35 Distributed Systems Middleware
Group

36. Probability of abort

Back-up Slides
11/28/2012 Leila Jalali
36 Distributed Systems Middleware
Group

37. Optimization1: Checkpointing and
Rollback mechanism
 Abort operation in multisimulations is not always
cost effective
 (i) long-running scientific simulation programs designed to run for
days need to restart from the beginning
 (ii) as the result of aborting one simulator, multiple dependent
simulators that used its updates need to restart as well, which
enforce very significant overhead on multisimulation execution
 We explore a checkpointing and rollback
mechanism [Elnozahy et al. 2002] that enables a
simulator to restart effectively from a checkpoint
 Computing the cost of roll-back:
Back-up Slides
11/28/2012 Leila Jalali
37 Distributed Systems Middleware
Group

38. Optimization 2: Relaxing
Dependencies

smok
t-bound=5 s
e
level
walking user
speed v-distance =3m speed
Num. of
ongoing n-update=5 Num. of
users
calls
11/28/2012 Leila Jalali
38 Distributed Systems Middleware
Group

39. A Case Study for simulation
integration
 To validate the proposed reflective architecture
 Using three disparate pre-existing simulators:
1. CFAST (Consolidated Model of Fire and Smoke
Transport): a fire simulator
 Simulates the effects of fire and smoke inside a building and
calculates the evolving distribution of smoke, fire gases and
temperature
2. Drillsim: an activity simulator
 Multi-agent system that simulates human behavior in a crisis
3. LTESim: a communication simulator
 Abstracts the physical layer and performs network level
simulations of 3GPP Long Term Evolution
11/28/2012 Leila Jalali
39 Distributed Systems Middleware
Group

40. Experiments: Integrating three simulators using
RAISE
2. Communication Simulator: LTESim
1. Evacuation Simulator: DrillSIM
Total num.
of evacuees
3. Fire Simulator: CFAST
some results
11/28/2012 Leila Jalali
40 Distributed Systems Middleware
Group

41. Case study- simulators properties
Evacuation Simulator Communication Fire Simulator
Simulator
DrillSim LTESim CFAST
 Simulates a Performs network level Simulates the effects of
response activity simulations of 3GPP LTE fire and smoke inside a
evacuation Event based building
Time stepped Open source (in Time stepped
Open source (in Matlab) Black-box (no access to
Java) Parameters: num. of source)
Agent based transmit and receive Parameters: building
Parameters: health antennas, uplink delay, geometry, materials of
profile, visual distance, network layout, channel construction, fire
speed of walking, num. model, bandwidth, properties, etc.
of ongoing call, etc. frequency, receiver noise, Output: temperatures,
Output: num. of etc. pressure, gas
evacuees, injuries, etc Output: pathloss, concentrations: CO2, etc.
throughput, etc.
11/28/2012 Leila Jalali
41 Distributed Systems Middleware
Group

42. Experiments
Intermediate Phase
Early Phase
Intermediate Phase
Late Phase
Early Phase
Late Phase
(a) simulation phase (no. of actions) (b) simulation phase (no. of actions)
OS: Optimistic CS: Conservative HS: Hybrid LS: Lock-
step scheduling
 (a) Average synchronization overhead
 (b) Total execution time in different simulation phases
 Total number of dependencies=100
 HS, initially, the cost of abort is small, so the strategy will
lean towards being optimistic/closer to OS in
performance as the number of actions increases, it Middleware
42 Distributed Systems
11/28/2012 Leila Jalali
Group

43. Experiments- dependencies
2400.53
1830.210
1243.008
1127.999
1082.647
903.162
975.581
710.081
639.099
320.073
315.492
27.288
36.873
27.263
35.655
 Synchronization overhead for different number of dependencies in the
late phase of simulation
 Overall performance of HS is better than CS and OS over a broad
range of dependencies
11/28/2012 Leila Jalali
43 Distributed Systems Middleware
Group

44. Experiments- different simulators
 Synchronization overhead and Total execution time of
different simulators CFAST, DrillSim, and LTEsim
 We need take into account the type of simulator and
number of dependencies
 Overhead in LTESim is very high when OS is used
 an event-based simulator with a large number of external
events that access the interdependent data items (e.g.
throughput) results in aborts
11/28/2012 Leila Jalali
44 Distributed Systems Middleware
Group

45. Experiments- checkpointing & rollback
 Drillsim (activity simulator) has been modified to
develop checkpointing mechanism
 using libckpt user level incremental checkpointing
 Results of optimistic scheduling (OS) and hybrid
scheduling (HS) with and w/o checkpointing
checkpointing overhead
11/28/2012 Leila Jalali
45 Distributed Systems Middleware
Group

46. Experiments- relaxations
Strategy CS OS HS
Metric Without With Without With Without With
Relaxations Relaxations Relaxations Relaxations Relaxations Relaxations
CFAST 592.228 510.376 778.364 353.856 292.283 201.598
DrillSim 404.293 393.634 312.182 296.341 104.293 98.781
LTEsim 946.432 883.812 1420.091 825.120 731.423 423.923
Total 1943.013 1787.882 2499.637 1475.317 1127.999 724.302
 Total number of relaxed dependencies= 10
 Relaxations always help into get better results in terms
of synchronization overhead and total execution time
11/28/2012 Leila Jalali
46 Distributed Systems Middleware
Group

47. Outline
 Motivation for New Simulation Platforms
 Reflective Architecture and Methodology
 Synchronization in Multisimulations
 Modeling Synchronization Problem using concepts from Multidatabases
 A Hybrid Scheduling Approach
 Optimizations: Checkpointing and roll-back; Relaxation models
 An Emergency Response Case Study
 Data Exchange in Multisimulations
 Modeling Data Exchange using Transformation Graphs
 Data Transformation Plan Generation: Algorithms
 An Emergency Response Use Case
 Formal Specification of Multisimulations
 Modeling Multismulations in Maude for rapid prototyping and analysis
 Conclusions and Future work
11/28/2012 Leila Jalali
47 Distributed Systems Middleware
Group

48. Data Exchange Problem
• How to exchange/tranform the information among simulators in
multisimulations execution
• Challenges:
– Data from a simulator is transformed into the meaningful input for another
simulator
– Different simulators represent data items differently (e.g. different geography
representation such as raster, network, etc.)
Harmful Agents Profile :
conditions Health
Nodes
Cell
Links CFAST Drillsim
48 Distributed Systems Middleware Group

49. Pipeline vs. Concurrent execution
Pipeline execution : Concurrent execution :
• Execute the simulators one at a time  Provide a framework in which
• Transform the output of a simulator into independent simulation models can
the input data of the other simulator execute concurrently
• Need data transformation, GIS  Need time synchronization to preserve
conversions, time adaptations, etc. casualty (e.g. scheduling)
 Need intermediate data transformations
based on dependencies, GIS conversions,
Simulation output etc.
Model 1 transformed
output
Simulation Simulation
Model i ... Model j
Simulation
Output
Model 2
intermediate
outputs .
.
Output
. synchronization
(a) Pipeline execution Simulation
Model n
(b) Concurrent execution
49
Distributed Systems Middleware Group

50. Our Goal
• Goal: During the concurrent execution of multiple simulators,
transform the data generated/updated by simulators into
data required by other dependent simulators
– minimize transformation cost at runtime
– retain flexibility to model/run multiple data exchange
scenarios
• Challenges:
– Dependencies among simulators may be many and complex
– Not necessarily a static transformation scenario: need the ability to
consider subset of dependencies to model/run different scenarios
50 Distributed Systems Middleware Group

51. Useful approaches for data exchange
Data Exchange in HLA Schema Mapping Ontologies
 A pub/sub model [4]  Data Exchange is an old, but recurrent,  Matching strategies for
- Based on routing spaces and database problem concepts, attributes
regions - Schema mappings and query answering in Clio and relations [43]
 express an interest in (IBM) [76]  Ontologies for GIS
receiving certain data  Transformation graphs in ETL process [83] (integrated geographic
or sending data by  Finding best queries through Steiner tree information systems)
defining subscription generation [81, 82] [91]
regions and update - Answering specific user queries (Q system)  Mapping discovery
regions - Answering relationship queries over entity-relation-  Comparing & Merging
 Interest management style data graphs (STAR) [39]
concepts using
platform (LUICID)  Semantic trees for relational DB schemas
ontologies (e.g. FCA-
- Supports dynamic multicast [80] , XML databases [77],
Merge, IFMAP, MAFRA
grouping  Data exchange & query answering for
frameworks) [93]
incomplete data sources [PDMS]
Data Exchange in S&M Data Exchange in workflows
 DEVS frameworks (Python DEVSIMPY, MATLAB/Simulink)  AXMEDIS for workflow management systems
 Model integration (using DSML)  Integration of multiple workflows (SESA)
51 Distributed Systems Middleware Group

52. Model Transformations
• Convertors can be combined to support a transformation
• One-to -one mapping from one data item in a simulator into a data item in
another simulator
• Modeled as a directed graph of convertors
• We assume that the transformations between two dependent data items are given to us
Transformations
Runtime
C2
Simulator i A C1 B Simulator j
C3
52 Distributed Systems Middleware Group

53. Transformation Graph
• One data item can involve in many dependencies
• We consider one-to-many mapping that captures all dependencies from one data
item in a simulator into data items in other simulators
• Generates a transformation graph:
– The vertices correspond to the various data items (source and target, intermediate)
– The directed edges between two vertices represents the connection that converts/transmit the data
– The associated conversion cost is represented by the weight of the edge
Runtime
C2
Simulator i d1 i d1 j Simulator j
C1
.
. C3 .
. .
.
D1i Source data dmj
C5
Intermediate data
.
.
D1j Target data .
C4
Directed edges: d1 k
C Convertors C7 Simulator k
1 .
C6 .
.
back-up slides Distributed Systems Middleware Group
53

54. Cost Model
54 Distributed Systems Middleware Group

55. Transformation problem
 Transform a single source data item to a single target data
item
 Consider one dependency between two data items at a time
 Transform a single source data item to all target data items
 Consider all the dependencies for a data item
 Transform a single source data item - multiple target data
items
 Consider some of the dependencies among multiple simulators
 Retain flexibility to model/run multiple data exchange scenarios
 Transform Multiple source data item - multiple target
55 Distributed Systems Middleware Group

56. Transformation (one-to-one)
• Transform a single source data item to a single
target data item
 Consider one dependency between two data items at a
time
 |N| = 2: Shortest path
Runtime
C2
Simulator i d1 i d1 j Simulator j
C1
.
. C3 .
. .
.
dmj
C5
.
.
.
C4
d1 k
C7 Simulator k
.
C6 .
.
56 Distributed Systems Middleware Group

57. Transformation (one-to-all)
 Transform a single source data item to all target
data items
 Consider all the dependencies for a data item
 N = V: minimum spanning tree
Runtime
C2
Simulator i d1 i d1 j Simulator j
C1
.
. C3 .
. .
.
dmj
C5
.
.
.
C4
d1 k
C7 Simulator k
.
C6 .
.
57 Distributed Systems Middleware Group

58. Transformation (one-to-many)
 Transform a single source data item - multiple target data
items
 Consider some of the dependencies from a data item in a simulator
to data items in other simulators
 NP-hard (the problem of computing the Minimum directed
Steiner Tree can be reduced to an instance of this
problem) Runtime
C2
Simulator i d1 i d1 j Simulator j
C1
.
. C3 .
. .
.
dmj
C5
.
.
.
C4
d1 k
C7 Simulator k
.
C6 .
.
58 Distributed Systems Middleware Group

59. Transformation problem: our focus
 Given a transformation graph find a
transformation plan with minimum total
cost from a single source data item to
multiple target data items
 NP-hard (the problem of computing the Minimum
directed Steiner Tree can be reduced to an instance
of this problem)
 in small transformation graphs
 Polynomial time algorithm for Steiner tree when we have
O(1) terminals [78]
 in large transformation graphs
 Shortest paths heuristic bases on Prim’s minimum
59 Distributed Systems Middleware Group

60. TR Problem in Small TG
• In multisimulations, in some of real world scenarios the
size of transformation graph and the number of target
data items for a specific source data item is relatively
small
• the experiments of soup membrane can be used to solve
the Steiner Tree Problem efficiently [78]
• Polynomial time algorithm for Steiner tree when we have
O(1) terminals:
– Enumerate all sets W of at most r – 2 non-terminals
– For each W, find a minimum spanning tree
– Take the best over all these solutions
60 Distributed Systems Middleware Group

61. TR Problem in Small TG (cont.)
Optimal TR for small transformation graphs
61 Distributed Systems Middleware Group

62. TR problem in large TG
• In multisimulations, in some scenarios the size
of transformation graph and the number of target
data items for a specific source data item might
be large
• Compute a minimum spanning tree of this
subgraph [83]
• Start with a subtree T consisting of one terminal
• While T does not span all terminals
– Select a terminal x not in T that is closest to a vertex in T.
– Add to T the shortest path that connects x with T
Back-up Slides
62 Distributed Systems Middleware Group

63. TR problem in large TG (cont.)
IterativeTR for large transformation graphs
1. Input: P: The initial plan, K: Number of iterations;
2. Output: P: The refined plan;
1. for all j=0 to k do
2. Ni= SelectNode(P);
3. RefinePlan(P,Ni);
4. return P;
RefinePlan
63 Distributed Systems Middleware Group

64. An emergency response usecase
smoke
wrapper
wrapper
cars demand CFAST
Drillsim
num. of
ongoing calls
cell-throughput building
geometry
wrapper wrapper
cell-throughput
TranSim LTESim
64 Distributed Systems Middleware Group

65. Geographies registered in the case study:
Simulator Type Entity Types Description
Raster Cell UCI outdoor region
Drillsim Raster Cell CalIT2 building floors
Raster Cell DBH building floors
TranSim Network Node, Link Road Network
LTESim GIS Point, Polygon Cell towers and throughput
Input and Output Parameters of simulators:
Simulator Type Data item type Geo. Type
Drillsim In Cell-throughput int (0-100) Cell
Occupancy int(>=0) Cell
Out Cars demand int(>=0) Cell
Num. of ongoing calls int(>=0) Cell
TranSim Cars demand int(>=0) Node
In
Cell-throughput int (0-100) Node
Cars int(>=0) Node
Out
Occupancy int(>=0) Link
LTESim In Num. of ongoing calls int(>=0) Point
Out Cell-throughput int (0-100) Polygon
65 Distributed Systems Middleware Group

66. Transformations Graphs
cell-to-polygon
walking mph-to-km/h user
C28
1 speed speed
C30 cell-to-polygon LTESim smoke-to-level
Drillsim harmful condition
C29 C21
4 smoke-to-health health
smoke
CFAST C21 C22 Drillsim
num. of cell-to-polygon
ongoing num. of
C28 users
2 calls polygon-to-cell
Drillsim
cell-to-polygon LTESim 5
pathloss C31 coverage
C29
polygon-to-cell
cell-to-node LTESim Drillsim
C32
cars demand C8 num. of cars throughput
3
TranSim
Drillsim
C9
cell-to-node 66 Distributed Systems Middleware Group

67. Transformations Results
Transformation Convertor Total num. Total time Avg. time Avg. Transformation Avg.TR Transformation time
ID (ms) overhead (ms) time (ms) (ms)
1 30 109 2.208 0.020 4.442 2.645
28 106 2.285 0.021
29 3 0.382 0.127
2 28 24 0.592 0.024 0.024 0.024
29 0 0 0
3 8 22 0.409 0.018 0.808 0.423
9 8 1.869 0.233
4 21 300 6.207 0.020 0.048 0.032
22 300 8.345 0.027
5 31 21 0.408 0.019 0.904 0.810
32 9 1.998 0.222
67 Distributed Systems Middleware Group

68. Experiment set up- large graphs
• The default value for α and β is set to one in the cost function
meaning that the communication and computation
normalized cost units are the same
• The effect of using optimal and heuristic TR algorithms in
reduction of transformation cost
• Generate random graphs with GT-ITM random graph
generator [23] (2500 nodes and 10660 edges with latency
between 10 to 90 ms)
• Measurements are averaged for 500 data exchange over the
complete multisimulation phase (over 3500 actions)
68 Distributed Systems Middleware Group

69. Experiment Results
69 Distributed Systems Middleware Group

70. Outline
 Motivation for New Simulation Platforms
 Reflective Architecture and Methodology
 Synchronization in Multisimulations
 Modeling Synchronization Problem using concepts from Multidatabases
 A Hybrid Scheduling Approach
 Optimizations: Checkpointing and roll-back; Relaxation models
 An Emergency Response Case Study
 Data Exchange in Multisimulations
 Modeling Data Exchange using Transformation Graphs
 Data Transformation Plan Generation: Algorithms
 An Emergency Response Use Case
 Formal Specification of Multisimulations
 Modeling Multismulations in Maude for rapid prototyping and analysis
 Conclusions and Future work
11/28/2012 Leila Jalali
70 Distributed Systems Middleware
Group

71. HLA and Multisimulations
Criterion HLA Multisimulations
Objective ─ Interoperability ─ Semantic Interoperability, Reusability, Flexibility
─ Reusability [24]
Formal
─ DEVS/HLA ─ Maude [27]
Specification
Domain ─ Defense ─ Flexible via use of domain ontologies [20]
Complexity ─ Low level knowledge
─ No need to conform the internal properties
needed
─ Semantic constraints implemented at the
─ Lack of semantic
metalevel [22]
interoperability
Time Management ─ Optimistic, Conservative, and Hybrid methods
─ Optimistic and conservative
[25?]
methods
─ Relaxed dependencies [23]
Data Exchange ─ A pub/sub model based on
─ Data exchange using transformation graphs
routing spaces and regions
Separation of ─ Merges domain-specific and
─ Separate concerns related to simulation domain to
Concerns integrated simulation
[20] Demo paper at Middleware 2009 those related to integration mechanisms [21] [26]
aspects
[21] ARM’09: Adaptive Reflective Middleware 2009
[22] DOS’10: Middleware Doctoral Symposium 2010
[23] Spring SIW’11: Simulation Interoperability Workshop 2011
[24] FHCT’11: Formal Modeling: Actors, Open Systems, Biological Systems 2011
[25] submitted to TOMACS’12: ACM Transactions on Modeling and Computer Simulation 2012
[26] Caltech Smart Grid Symposium 2011
[27] TMS/DEVS’11: Theory of Modeling and Simulation 2011
11/28/2012 Leila Jalali
71 Distributed Systems Middleware
Group

72. Conclusion and Future work
 Contributions:
1. Architecture for simulation integration
 A methodology to develop multisimulations
2. Time synchronization
 Hybrid approach for synchronization
 Relaxation model
3. A Data Exchange Use-case
 TR graph and algorithms
4. Formal specification using Maude capabilities
 Analyze the scenarios in a cost effective development
 Future work:
 Data Transformations and Ontologies
 Adaptive Checkpointing Protocol
 End User Experience
11/28/2012 Leila Jalali
72 Distributed Systems Middleware
Group

73. Thanks
jalalil@uci.edu
http://www.ics.uci.edu/~ljalali/
Distributed Systems Middleware
11/28/2012 Leila Jalali Group

74. Back-up slides
Distributed Systems Middleware
11/28/2012 Leila Jalali Group

75. Checkpoiting protocol

Back to main slide
11/28/2012 Leila Jalali
75 Distributed Systems Middleware
Group

76. Checkpoiting protocol (cont.)
Back to main slide
11/28/2012 Leila Jalali
76 Distributed Systems Middleware
Group

77. Rollback using checkpoints
 Restart from a checkpoint • S2 restarts from t=0
log • replay all incoming
messages using log
S1 S2 S3 t=0 • apply the update from S1 at
t=8
• continue the execution
chki chkj
using checkpoints:
t=8 • S2 restarts from checkpoint chki
t=8 • replay all incoming messages using log
t=10
until t=8 then apply the update from S1
t=11 • which checkpoint? the most recent
checkpoint before t=8
Back to main slide
11/28/2012 Leila Jalali
77 Distributed Systems Middleware
Group

78. Simulator restarts
 S2 aborts because of the update received from S1
 S3 aborts because of the update received from S2
S2 • S2 restarts from t=0 S3 • S3 restarts from t=0
• replay all incoming • replay all incoming
log
messages using log messages using log
• apply the update from S1 at • apply the update
t=8 from S2 at t=10
update from S1
• continue the execution • continue the
t=8 execution
t=10 t=10
Back to main slide
11/28/2012 Leila Jalali
78 Distributed Systems Middleware
Group

79. Proof of reduction
 Membership of ST in NP: trivial
 Hardness: take instance G=(V,E), k of Vertex
Cover
 Build G’ by subdividing each edge
 Set N = set of new vertices
 All edges length 1
 G’ has Steiner Tree with |E|+k – 1 edges, if
and only if G has vertex cover with k vertices
Back to main slide
11/28/2012 Leila Jalali
79 Distributed Systems Middleware
Group

80. Approximation algorithms
 Several different algorithms that guarantee ratio 2
(or, more precise: 2 – 2/n).
 Shortest paths heuristic
 Ration 2 – 2/n (no proof here)
 Bases on Prim’s minimum spanning tree algorithm
 Start with a subtree T consisting of one terminal
 While T does not span all terminals
 Select a terminal x not in T that is closest to a vertex
in T.
 Add to T the shortest path that connects x with T. Back to main slide
11/28/2012 Leila Jalali
80 Distributed Systems Middleware
Group

81. Data Transformations in concurrent
execution

Back to main slide
11/28/2012 Leila Jalali
81 Distributed Systems Middleware
Group

82.  Pyramid of specifications:
 Observe-Analyze-Adapt cycle
 (i) Observe changes in desired attributes of independently
executing simulations at runtime (reification)
 (ii) Analyze and evaluate whether changes in the observed
attributes impact parameters in other simulators
 (iii) Adapt the execution of the multisimulation by enforcing the
changes back into the impacted simulators (reflection)
11/28/2012 Leila Jalali
82 Distributed Systems Middleware
Group

83. Using Formal Specifications
b. System analyzer
a. developer try changes
Formal
Formal methods specification
make changes
tool Reasoning engine
Meta level
run-time
meta-models
inter-dependencies
Base level
Base level
simulator1 simulator2 simulator3 Live data simulator1 simulator2 simulator3 Live data
preprocessing steps run-time execution
Using Maude specification by a developer to
answer questions at design steps:
- What kind of simulator we need?
- e.g. Micro vs. Macro traffic simulator
Maude can help to see the scenarios before the real
integration which results in a cost effective
Back-up Slides
development
11/28/2012 Leila Jalali
83 Distributed Systems Middleware
Group

84. Modeling Multisimulations in Maude
Multisimulations concept Maude concept
A simulator Maude class
(inheriting from Oid-Object)
Simulator object Maude object
Simulator type Maude rewrite rules
Multisimulation configuration Maude configuration
Updates, acks Maude messages
Behaviour specification Rewrite rules
Back to main slide
11/28/2012 Leila Jalali
84 Distributed Systems Middleware
Group

85. The Maude Rewriting Language
 Developed at SRI
 Maximizes simplicity (only equations and rules), Performance
and Expressiveness ( Equation pattern matching, User-definable
syntax and data)
 Features
 High performance engine
 Modularity and parameterization
 Reflection -- using descent and ascent functions
 Search and model-checking
 Full Maude: > maude.linux full-maude.maud
Back to main slide
11/28/2012 Leila Jalali
85 Distributed Systems Middleware
Group

86. Using FS to choose a simulator
2. What kind of simulator is appropriate?
Developer
(integrator) Macro traffic simulator?
- tend to model traffic
Add a third simulator as a continuous flow
Muade Specifications
op smokeLevel : Sim-Oid Syn-Oid Int Int -> Msg [ctor] .
rl[FS] :
Micro traffic simulator?
[fs : FS | syn: asyn, smoke: n, temp: t]
=> [fs : FS | syn: asyn, smoke: n, temp: t] smokeLevel(fs,syn,n,t) .
- capture the behavior of
Meta level
rl[SC] : Traffic Simulator
smokeLevel(fs,syn,n,t)
….
vehicles and drivers in
great detail, including
interactions
among vehicles, lane
Base level
changing, etc.
Activity Fire
Simulator Simulator
(Drillsim) (CFAST)
Back to main slide
11/28/2012 Leila Jalali
86 Distributed Systems Middleware
Group

87. Using FS to choose a simulator
2. What kind of simulator is appropriate?
(1) Macroscopic (macro) models (e.g. Strada, Metacor):
Low
tend to model traffic as a continuous flow, often
using formulations based on hydrodynamic flow
level of details
theories.
(2) Mesoscopic (meso) models (e.g. DynaMIT,
DYNASMART) : model individual vehicles, but at an
aggregate level, usually by speed-density relationships
and queuing theory approaches.
High
(3) Microscopic (micro) models: (e.g. MITSIMLab
,Vissim) :capture the behavior of vehicles and drivers in • Using Micro simulators can be very
great detail, including interactions among vehicles, lane time consuming and tedious
because of the detailed nature of
changing, response to incidents, and behavior at
the models, the preparation of
merging points. Add a third simulator to the scenario input data (e.g. network coding and
representation) - do we really need
it?
Fire
Activity
Simulator Traffic Simulator • Macro simulators do not have
Simulator
(syn) (fs) (?) ability to capture the detailed
behavior needed to study traffic
87 Distributed Systems Middleware
11/28/2012 Leila Jalali networks with dynamic traffic
Group

88. Using FS to add a relationship
Muade Specifications
 Modeling the relationships between
op smokeLevel : Sim-Oid Syn-Oid Int Int -> Msg [ctor] .
rl[FS] :
existing simulators
[fs : FS | syn: asyn, smoke: n, temp: t]
=> [fs : FS | syn: asyn, smoke: n, temp: t] smokeLevel(fs,syn,n,t) .  Analyze the parameters sending
Meta level
rl[SC] :
smokeLevel(fs,syn,n,t)
….
between the simulators using Maude
rules
 Decide if we need a new relationship
Base level
to bring into the scenario
Activity Fire Communication
Simulator Add a new relationship
Simulator Simulator
(syn) (fs) (com)
numUsers (syn,com,n,t)
time-stamp Number of people using
Num. of people
cellphones in activity simulator
smokeLevel(fs,syn,n,t) as a parameter that effects
smoke number of users in the
time-stamp communication simulator
Back to main slide
11/28/2012 Leila Jalali
88 Distributed Systems Middleware
Group

89. Multisimulation configuration

11/28/2012 Leila Jalali
89 Distributed Systems Middleware
Group

90. Configurations transitions

Back to main slide
11/28/2012 Leila Jalali
90 Distributed Systems Middleware
Group

91. Using capabilities of Maude
 As the multisimulation system grows, consistency checks
become more important
 Using the reflective capability of Maude to formalize
metadata and carry out various meta-analyses
 meta-reduce: returns the representation of fully reduced from
a term using the equations as a parameter.
 meta-apply: simulates the application of a given rewrite rule to
a term
 Explore all the behaviors from an initial state up to
termination
 Checking that in all execution sequences all messages
arrive to their destinations
 Information justifying or qualifying a rule and rate
information about a rule Back to main slide
 Adapt information exchange91
11/28/2012 Leila Jalali
Distributed Systems Middleware
i.e. granularity of data being
Group