2. Goals For Today!
Multiple Variable Models!
Simple Birth Rates Revisited !
Automation III --- !
" " "The Behavior Space!
3. Multiple Variable
Models!
• We’ve seen that Netlogo
provides several ways to output
data and run the model so that
we can get statistical
information.!
4. Multiple Variable
Models!
• What if there are several
variables in a model? Will this
increase the time it would take
to test the parameter space?!
5. Simple Birth Rates!
• Open the Simple Birth rate
model!
• There are three variables,
carrying capacity, blue fertility
rate, and red fertility rate.!
!
6. Simple Birth Rates!
• The Blues and Reds reproduce
according to their fertility rate
and the entire population is
limited by the carrying capacity.!
7. Simple Birth Rates!
• There is already an output
window on the bottom of the
screen that will show how long
it took for one of the colors to
go extinct.!
• Use the “run experiment” button
to start the model. Run the
model a few times, varying the
fertility rates.!
8.
9. Simple Birth Rates!
• The output is relatively simple,
but it would probably take a
long time to move through each
combination of blue and red
fertility rates, especially if you
wanted to run the model more
than once for each combination. !
10. Simple Birth Rates!
• We could also vary the carrying
capacity, which would also
increase the amount of time it
would take to run these models
by hand.!
13. What is BehaviorSpace?!
• “BehaviorSpace is a software
tool integrated with NetLogo
that allows you to perform
experiments with models.”!
!
!
• http://ccl.northwestern.edu/
netlogo/docs/behaviorspace.html !
14. What is BehaviorSpace?!
• “Behavior Space runs a model
many times, systematically
varying the model's settings and
recording the results of each
model run. This process is
sometimes called "parameter
sweeping”!
15. What is BehaviorSpace?!
• It lets you explore the model's
"space" of possible behaviors
and determine which
combinations of settings cause
the behaviors of interest.”!
16. Why Behavior Space is
Useful!
• BehaviorSpace automates the
movement of the model runs
through each combination of
variables, allowing us to move
quickly across the parameters
we wish to test.!
17. Why Behavior Space is
Useful!
• It also outputs the information
into a .csv file so that we can
use statistical software to
analyze the data.!
18. Why Behavior Space is
Useful!
• So if you have a lot of model
runs, multiple variables, or
want to analyze model data with
statistical software, then
BehaviorSpace may be of use to
you!!
19. Why Behavior Space !
is Useful!
• Let’s see how BehaviorSpace
works.!
!
• Go to the Tools scroll down
menu and click BehaviorSpace.!
20.
21. Working with the !
Behavior Space!
• Now you should have the
experiment page.!
!
• You will see the options New,
Edit, Duplicate, Delete, and Run.!
22.
23. Working with the !
Behavior Space!
• The New button creates a new
experiment, and brings you to
the page where you will be able
to specify the parameters of the
model you wish to test and how
many times you wish the model
to run per combination.!
24. Working with the !
Behavior Space!
• The Edit button allows you to
edit any experiment that is saved
in the BehaviorSpace experiment
list. !
!
25. Working with the !
Behavior Space!
• If you save your Netlogo
program, it will also save your
experiments, so you do not have
to write out the parameters you
wish to test each time you turn
off Netlogo.!
26. Working with the !
Behavior Space!
• The Duplicate button allows you
to create another copy of an
experiment that will be saved in
the experiments box of
BehaviorSpace.!
27. Working with the !
Behavior Space!
• The Delete button permanently
deletes an experiment from the
experiments box.!
28. Working with the !
Behavior Space!
• The Run button begins the
process of running your
experiment and outputting the
data into a file.!
29. Experiment Setup!
• Let’s go to the experimental setup!
• Click the ‘New’ button.!
• You should now see a screen that
will allow you to detail the
variables you wish to test and
how the program should run the
experiment.!
30.
31. Setting up an
Experiment!
• The Experimental setup screen
has a title box, a variables box,
an area that allows you to
control how many times the
model should run per
combination, and boxes that
specify any commands you wish
the program to run when the
setup or go button is pressed.!
!
32. Setting up an
! Experiment!
• There are also boxes that allow
you to detail any conditions
that should stop the running of
the program and any commands
that should go into effect at
the end of the model run.!
• It also includes the ability to
determine a limit to how many
steps the model will run for.!
33. Setting up an
Experiment!
• Let’s take an in depth look at
each of these features.!
• I will give the title “Experiment
#1: Blue & Red Fertility [0 1
10]” to my experiment and move
on to the variables box.!
34.
35. Variables Box!
• The variables box includes
variables that come from
sliders, switches, and choosers
(draw down menus) on the
interface. It can also include
variables found within the
program code.!
36. Variables Box!
• The user is be able to specify
the boundaries of the parameter
space they wish BehaviorSpace
to “sweep” through. This is done
by writing in the variables and
the values the user wishes to
test.!
37. Variables Box!
• BehaviorSpace keeps variables
higher up in the box constant as
it cycles through the lower
variables’ value settings, only
moving the higher variables to
their next setting after
finishing a complete cycle
through the possible
alternatives in the variables
below it.!
38. Variables Box!
• If the runs are taking place in
parallel, the output may not
exactly mirror this process.!
• Regardless of the case, no
matter where you locate a
variable, the entire parameter
space for all the variables you
define in the box will be tested.!
39.
40. How to’s!
• Assigning a value to a variable:!
• Ex: ["blue-fertility" 10] This will
give the variable blue-fertility
the value of 10 in all of the
model runs.!
41.
42. How to’s!
• Assigning two values to a
variable (listing values)!
!
• Ex: ["blue-fertility" 1 2] gives the
variable blue-fertility the value
of 1 and runs through all other
combinations (if there are other
variables), and then moves on to
2 and runs through all the
other possible combinations
with red-fertility and carrying
capacity.!
43.
44. How to’s!
• Assigning multiple values to a
variable (listing values)!
• Ex: ["blue-fertility" 1 2 4 7] Runs
through 1 and 2 and also does
4 and then 7. !
45.
46. How to’s!
• Using an interval to assign
multiple values!
• The interval must be inside a set
of brackets!
!
47. How to’s!
• Ex: ["blue-fertility" [1 1 3]] This
example runs through the
values 1 through 3, moving at
an increment of 1. Therefore, it
will assign the value 1, 2, and 3
to the variable as it moves
through the different
combinations.!
!
48. How to’s!
• When assigning values to a
variable with an interval, be
careful to only include the
numbers you wish to test. The
interval is inclusive and will test
all the numbers you specify.!
• Ex: ["blue-fertility" [0 1 4]] This will give the
variable the values 0, 1, 2, 3, and 4.!
49. How to’s!
• Behavior Space is entirely
inclusive of the specified range!
• In other words, Inclusive will
execute all Parameter Combos in
the Range!
!
50.
51. How to’s!
• Repetitions: The user can select
how many times Netlogo should
run a given combination of
variables!
52.
53. How to’s!
• Measure runs using these
reporters: This is what the
model will measure and output
in the data.!
• Measure runs at every step:
Checking this means the data
will include output for each
step in the model!
54.
55. How to’s!
• Setup Commands: The user can
include commands that will be
executed in addition to calling
the ‘to setup’ procedure when
the setup button is hit.!
56. How to’s!
• Go Commands: The user can
include commands that will be
executed in addition to calling
the ‘to go’ procedure when the
go button is hit.!
57.
58. How to’s!
• Stop Conditions: This box
allows the user to specify
conditions that would end the
model run if they were met. !
!
59. How to’s!
• While there may be many reasons
a modeler would want to do
this, one reason could be to
prevent illogical or undesirable
combinations of variables from
occurring while the program
sweeps through parameter
space. !
60.
61. How to’s!
• Final Commands: The user can
specify commands to run at the
end of a model run.!
62.
63. How to’s!
• Time Limit: The user can specify
how many ticks a given
combination of variables will be
allowed to run before stopping.!
• In certain instances model run
will go on forever … need to
have a time limit in that case!
64.
65. Simple Birth Rates in
the Behavior Space!
!
• OK, now let’s run the model. We
will put a time limit of 100 ticks
for each model run.!
67. Running BehaviorSpace!
• Select the experiment we just
created and hit “Run”.!
• You will now be given the option
to select if your data will be
put in a spreadsheet or in a
table.!
• The data file will be saved as a
comma separated values (.csv)
file.!
68. Parallelization!
• You will also be given the option
to choose how many experiments
should run in parallel.!
• By default, Netlogo will run
one experiment for each core
processor in your computer.!
69. Parallelization!
!
• Only one experiment will be
shown on screen, the others
will be run in the background.!
• You should individually test
what your computer can handle,
since many parallel runs can
slow down your computer.!
70.
71. Increasing the Speed!
• After making those choices the
model will begin to run.!
• You will have the option to turn
off viewing the plot and
updating the visualization,
plots, and monitors. This will
help increase the speed of the
runs.!
72. Increasing the Speed!
• You can also use the speed
slider, which determines how !
many times the image should be
updated.!
73.
74. Output!
• Spreadsheet data!
!
• At the top, the data sheet will
include the model and experiment
name, and the size of the world.!
• The commas will allow you to
demarcate where the columns
should be in the program you will
use to analyze the data.!
75.
76. Output!
• Tables (might be easier to work with in
outside statistical software)!
!
• At the top, the data sheet will
include the model and experiment
name, and the size of the world.!
• The commas will allow you to
demarcate where the columns
should be in the program you will
use to analyze the data.!
77.
78. Output!
• You will probably want to edit
the file before sorting it. The
information at the top can be
copied and pasted elsewhere if
you need it. !
• The actual output from the
model runs will be below.!
79.
80. Output!
• Once you have edited your file,
you will be able to sort the data
as you wish. !
"Example in Excel Data --> TexttoColumns!
"Then select delimited and by comma!
!
• As we saw before, the output may
not be in the order, since many
runs may have been running in
parallel.!
81. Output!
• Once your data is in a useful
format, you can begin to analyze
it! !
82.
83. In Summary!
• We have seen that
BehaviorSpace can run through
the parameter space much
faster than having a human go
by hand through all the
different variable combinations.!
84. In Summary!
!
• The previous data file took a
little more than 5 minutes to
create and output.!
85. There are Limitations!
• But it moved in increments of
1.0 and the carrying capacity
remained constant across the
different combinations. Most
importantly, the model was
limited to 100 ticks, which may
not be enough time to witness
the dynamics the model is
supposed to display.!
86. There are Limitations!
• By adding these other features
we will see how quickly the
length of time required to run
the models would grow if we
added smaller increments of
movement across the parameter
space and the extra carrying
capacity variable.!
88. Limitations!
• 40,814,201 runs is a lot!
• Let’s assume (probably
incorrectly) that it takes 1 sec /
run (Including all the steps in
one run of the model)!
• 40814201 / 60 secs / 60 mins /
24 hours / 7 days = 67.484
weeks, 1.298 years!
!
89. Limitations!
• That’s not including parallel
runs, but even if you could run
100 in parallel, it would still
take a considerable amount of
time!
• Thus, full parameter sweep
Probably Not feasible in
Netlogo!
90. Limitations!
• One option would be to limit the
parameter space to an area of
specific interest. !
• For example, the dynamics of
two competing groups with
varying fertility rates is not
particularly interesting when
the carrying capacity is zero
from the outset. !
91. Limitations!
• Netlogo was mainly designed for
visualizations and as an
introductory language.!
• If you write good modular code,
you will be able to convert your
model into another language that
can perform massively parallel
runs more efficiently than
Netlogo (e.g. Python or C++)!
