Presentation held at the 17th Annual Workshop on Economic Heterogeneous Interacting Agents WEHIA 2012, Paris, June 21-23, 2012. Abstract: Agent-based approaches are getting more and more attention recently. In our current work we replicated the initial model of Domenico Delli Gatti et al. described in their work entitled Macroeconomics from the Bottom-up. We address the question of validating and verifying simulations in general, also in the context of economic modelling, and summarize the lessons we learnt from the replication of the aforementioned model. The results highlight the importance of explicit documentation of the actors, timing of the events, and partial results that replicate the hallmarks of the model which can be verified independently with a set of simulation runs. You can find details in the paper included in the electronic conference proceedings!

1. Richárd O. Legéndi, László Gulyás
Eötvös Loránd University, AITIA International, Inc
legendi@inf.elte.hu, lgulyas@aitia.ai
Supported by the Hungarian Government (KMOP-1.1.2-08/1-2008-0002), the European Union Seventh
Framework Programme FP7/2007-2013 under grant agreement CRISIS-ICT-2011-288501 (CRISIS –
Complexity Research Initiative for Systemic InstabilitieS) and mOSAIC 2011-256910 (Open-Source API
and Platform for Multiple Clouds). These supports are gratefully acknowledged.
17th Annual Workshop on Economic Heterogeneous Interacting Agents
WEHIA 2012, Paris, June 21-23, 2012

2. Layout
 Motivation and Background
 The original Model
 Results
2012.06.22. WEHIA 2012 - Replication of MacroABM 2

3. 2012.06.22. WEHIA 2012 - Replication of MacroABM 3

4. Replication? Why care?
 Replication of experiments, validation of results are
essential
 „Simulations as experiments”
 If cannot be reproduced, its scientific value is in
question
 Models never replicated - except a few classical ones
 Helps us get a deeper understanding
 Of relevant properties, key issues
 Deploy simulation as a research tool
2012.06.22. WEHIA 2012 - Replication of MacroABM 4

5. Validation?
 Docking – alingment of different models
 Different computational models for the same
phenomenon
 Replication
 W/o being able to replicate results of an artificial
model, how to target real-world systems?
 Several problems, e.g. ambiguity
 Different approaches exist (AgentUML, ODD, etc.)
 But there’s no consensus on using them...
2012.06.22. WEHIA 2012 - Replication of MacroABM 5

6. Why to replicate? Parameter sweeps
„[...] suppose that in a model there are just 10 relevant
parameters, and that each parameter can assume 10
different values (a rather simplifying assumption). As a
result, one obtains that the constellation of the
parameter space is given by 10^10 vectors. If we perform
20 different runs for each one of them to take into
account the possible effects of changing the random
seeds, the total number of simulations would amount to
2*10^11!”
Gatti, Domenico Delli, Saul Desiderio, Edoardo Gaffeo, Pasquale Cirillo, and Mauro Gallegati:
Macroeconomics from the Bottom-up. 1st ed. Springer, 2011 (p. 76., section 3.10.1)
2012.06.22. WEHIA 2012 - Replication of MacroABM 6

7. Why to replicate?
 In a different environment?
 Matlab  Java/Mason
 Efficiency
 Reduce required time for a single simulation run
 Tool support: MEME
 Parameter sweep exploration
 Being Strong
 Exploiting Grid/Cloud systems
 Being Smart
 Design of Experiments
2012.06.22. WEHIA 2012 - Replication of MacroABM 7

8. Background
“The CRISIS project addresses building a next generation
macroeconomic and financial system policymaking model: a
bottom-up agent-based simulation that fully accounts for the
heterogeneity of households, firms, and government actors. The
model will incorporate the latest evidence from behavioral
economics in portraying agent behavior, and the CRISIS team will
also collect new data on agent decision making using
experimental economics techniques. While any model must
make simplifying assumptions about human behavior, the CRISIS
model will be significantly more realistic in its portrayal of relevant
agent behavior than the current generation of policymaking
models.”
Crisis project description: https://www.crisis-economics.eu/
2012.06.22. WEHIA 2012 - Replication of MacroABM 8

9. Replicated
Model
Modelling Economic Simulator
Framework (Cloud-Based
Parameter Sweep
Execution)
Models Web-based Game
(Participatory
Experiments)
2012.06.22. WEHIA 2012 - Replication of MacroABM 9

10. 2012.06.22. WEHIA 2012 - Replication of MacroABM 10

11. The Original Model
Source: Domenico Delli Gatti, personal communications
2012.06.22. WEHIA 2012 - Replication of MacroABM 11

12. Agents
 Households
 supply labor
 buy consumption goods
 hold deposits
 Firms
 demand labor
 produce and sell consumption goods
 Bank
 receive deposits from households
 extend loans to firms
2012.06.22. WEHIA 2012 - Replication of MacroABM 12

13. Market Processes I
1. Fims compute net worth, production/price and
labour demand
2. Credit market:
1. Bank decides credit conditions
2. Firms decide to whether take loan or not
3. Job market:
1. Firms redefine labour demand, publish vacancies:
1. Excess workforce: fire workers
2. Insufficient workforce: hire if possible
2012.06.22. WEHIA 2012 - Replication of MacroABM 13

14. Market Processes II
4. Consumption goods market:
1. Workers get wages and compute consumption budget
2. Firms post their price
3. Consumers contact z firms randomly
 Ordered by price
4. Unspent money  Involuntary savings
5. Unsold goods  Sold at zero cost (non-durable)
5. Accounting
1. Firms calculate profits
2. Earnings are retained profits
 Used to update net worth.
2012.06.22. WEHIA 2012 - Replication of MacroABM 14

15. 2012.06.22. WEHIA 2012 - Replication of MacroABM 15

16. Results I - Benchmarking
2012.06.22. WEHIA 2012 - Replication of MacroABM 16

17. Result II – Verification
 Scaled agents (w/o changing overall ratio)
 Up to 7500 agents
 Avg’d 40 runs
 T = 1000 time steps
 Included initial state
 High oscillations
 Until spontaneous
order emerges
(„equilibrium”)
2012.06.22. WEHIA 2012 - Replication of MacroABM 17

18. 2012.06.22. WEHIA 2012 - Replication of MacroABM 18

19. Summary
 We created a replication the MacroABM model in a
different environment
 Identic output
 Results are platform, environment-independent
 Opens up the window of standardized simulation tools
 Extensive parameter space explorations
 Performance speedup
 By the factor 5x-10x
 On the other hand, code length is increased similarly:
 Matlab: ~300 LoC
 Java: 1500 + 1000 LoC
2012.06.22. WEHIA 2012 - Replication of MacroABM 19

20. Download!
http://www.crisis-economics.eu/jmark-i-build-report
2012.06.22. WEHIA 2012 - Replication of MacroABM 20

21. Richard O. Legendi
legendi@inf.elte.hu
http://people.inf.elte.hu/legendi/
June 22., 2012.
Crsisis Website: http://www.crisis-economics.eu/