3. Levels and dynamics of
immergence
Norms may “sink” into
the mind
Various levels
Full norm processing
Violation/
Internalization compliance
• Internalized N-goal Full norm processing
• Independent of norm
enforcement belief N-internalization
• Internalized goal
• Independent of N--belief N-incorporation
Incorporation
• Automated behaviour
Bidirectional dynamics
4. A non-rigid modular
architecture
NORM
RECOGNITION
INPUT N-B1 NORM DECISION: CONFORM
B-B2 ADOPTION: BEHAVIOR
N-
N-B3 N-GOAL
INTENTION
Socrates
acceptance of
sentence
Internalization
5. A non-rigid modular
architecture
NORM
RECOGNITION
INPUT N-B1 NORM DECISION: CONFORM
B-B2 ADOPTION: BEHAVIOR
N-
N-B3 N-GOAL
INTENTION
Stop at red light
Mind
commands Answer requests if
eye contact is
established
Internalization Incorporation
7. Classic and current views
Classic conditioning: external sanctioning system never
completely abandoned (Scott 1971).
Internal sanctions (Parsons, 1952; Scott, 1971; Axelrod,
1986, Elster, 1989; Coleman, 1990; Dequech, 2009; etc):
Related to emotions of discomfort, shame and guilt
Sense of duty
Thoughtless conformity:
The more agents have conformed to a norm in the past,
the more they will redo it in the future (Epstein, 2007).
Why, when, how???
What about retrieving deliberation in norm-compliance?
8. In EMIL follow-ups
(Conte et al., 2011; Villatoro et al., 2010)
Formulate hypotheses about
effects, factors, and evolution of internalization
reversible dynamics
Implement essential mechanisms in EMIL-A.
No emotions at this stage
Check hypotheses by simulation
9. Internalized N-goal:
goal no more
relativized to N-
EMIL-Internalizer
enforcement belief.
N-enforcement belief
(EMIL-IA)
still persists but agent
does not calculate it I N-bel1:general
when decide to n
execute N-goal Internalized
p N-bel2:pertinence N-goal
Internalized goal: u
t
goal no more N-bel3: enforcement
internalized to
normative belief in
the general form. I N-bel1:general
Internalizer has lost n
normative origin of p Internalized
N-bel2:pertinence goal
her goal. u
t
N-belief still persist N-bel3: enforcement
but agent pursues
goal irrespective of it.
10. Internalized N-goal:
goal no more
relativized to N-
EMIL-Internalizer
enforcement belief.
N-enforcement belief
(EMIL-IA)
still persists but agent
does not calculate it I N-bel1:general
when decide to n
execute N-goal Internalized
p N-bel2:pertinence N-goal
Internalized goal: u
t
goal no more N-bel3: enforcement
internalized to
normative belief in
the general form. N-bel1:general
Internalizer has lost I
normative origin of n Internalized
N-bel2:pertinence goal
her goal. p
u
N-belief still persist t N-bel3: enforcement
but agent pursues
goal irrespective of it. Bel4
11. Factors of internalization
Why form a normative goal irrespective of external
sanctions?
Several factors, among which
Consistency with previously internalized norms
(McAdams, 2008) and temporal discount (see below)
“Internal sanctions” (Scott, 1971; Axelrod, 1986, etc).
12. Consistency with previous
norms
Why observe norms irrespective of
external sanctions?
Consistency (McAdams, 2008),
Successful educational strategies (King,
2008) link new inputs to previously
internalized norms
Policymaking: Controversial efficacy of
frightening announcements (Goodall,
2005): Inability to act under
representation of delayed reward due to
hyperbolic discounting (Bickel and
Johnson, 2003; Rachlin, 2000), i.e. effort
hyperbolically decreasing with time-
distance from goal-attainment,
• leads to procrastination
• Better refer to consistent norms and
values
13. Internal sanctions
“ The internal sanction of duty is a feeling in our mind; a pain, more or less
intense, attendant on violation of duty; it is the essence of conscience.”
(John Stuart Mill)
When. ground the mechanism on salience.
How:
Anticipatory representations of consequences (supported by
emotions)
tend to replace exter:al sanctions.
Why: the higher the salience,
The more the sanctioners
The more frequent and severe the sanction is observed/received
The more justified the sanction is perceived.
The more often it is expected
The more often it is anticipated, simulated in imagination
But the confine between
what is experienced in anticipating future sanctions
current sanction becomes fuzzier over time…
14. Internal sanctions
“ The internal sanction of duty is a feeling in our mind; a pain, more or less
intense, attendant on violation of duty; it is the essence of conscience.” (John
Stuart Mill)
When. ground the mechanism on salience.
How:
Anticipatory representations of consequences (supported by
emotions)
tend to replace external sanctions.
Why: the higher the salience,
The more the sanctioners
The more frequent and severe the sanction is observed/received
The more justified the sanction is perceived.
The more often it is expected
The more often it is anticipated, simulated in imagination
But the confine between
what is experienced in anticipating future sanctions
current sanction becomes fuzzier over time…
15. Internal sanctions
“ The internal sanction of duty is a feeling in our mind; a pain, more or less
intense, attendant on violation of duty; it is the essence of conscience.” (John
Stuart Mill)
When. ground the mechanism on salience.
How:
Anticipatory representations of consequences (supported by
emotions)
tend to replace exter:al sanctions.
Why: the higher the salience,
The more the sanctioners
The more frequent and severe the sanction is observed/received
The more “justified” the sanction is perceived.
The more often it is expected
The more often it is anticipated, simulated in imagination
But the confine between
what is experienced in anticipating future sanctions
current sanction becomes fuzzier over time…
16. Internal sanctions
“ The internal sanction of duty is a feeling in our mind; a pain, more or less
intense, attendant on violation of duty; it is the essence of conscience.” (John
Stuart Mill)
When. ground the mechanism on salience.
How:
Anticipatory representations of consequences (supported by emotions)
tend to replace exter:al sanctions.
Why: the higher the salience,
The more the sanctioners
The more frequent and severe the sanction is observed/received
The more justified the sanction is perceived.
The more often it is expected
The more often it is simulated in imagination
But the confine between
what is experienced in anticipating future sanctions
What is experienced during the effective sanction
becomes fuzzier over time…
17. Dynamics of
N-internalization
N- Delibe
inter rative
N-salience N-salience N-
naliz compli
ation ance
18. Dynamics of
N-internalization
N- Delibe
inter rative
N-salience N-salience N-
naliz compli
ation ance
19. Natural experimental
hypotheses
Natural experiments
Internalized goals lead to a more vigorous goal-attainment (Bargh
et al., 2001) than originally inner goals because of effect of
prospect theory (Kahneman and Tversky, 1979; Abdellaoui et
al., 2007)
Internalizer shows higher intolerance for norm violation based on
confirmation bias (Nickerson, 1998; Sternberg, 2007)
Internalizers are much better at defending the norms than
externally-enforced observers based on cognitive dissonance
(Festinger 1957).
However, internalizers are sensitive to salience
20. Artificial experimental
hypotheses
Artificial experiments
Compliance is more robust if norms are
internalized because independent of
external sanctioning entities.
Less costly, allowed evolution
Sensitivity to salience makes internalization
flexible and allows for norm innovation
21. Effects of internalization in artificial societies
(Villatoro et al., 2010)
• Three interacting subpopulations :
• Strategic
• Normative (EMIL-A)
• Internalizers (EMIL-IA)
• NA may internalize the N.
22. Hypotheses
• Strategic and Normative agents lead to a
collapse of cooperation with low punishment
rates
• Internalizers maintain social order imposed by
social norms even if these are not defended
• Temporarily, allowing the system to recover from
possible failures;
• Permanently: when salience decreases internalizers
start dismissing the norm and do cost-to-benefit-
calculation
23. Experimental scenario
• Agents play a repeated Prisoner’s Dilemma.
• Norm specifies “Cooperation”.
• Agents can punish those that do not follow the norm,
with a given probability.
• Depending on salience value:
• Strategic punishment: Fine.
• Normative punishment: Fine + Deontic Message.
25. Details
• Strategic Agents:
• Do not know about punishment. They only have a
benefit cost global perception of the interaction
• Normative Agents:
• They are initialized with a risk-taking value. They will
defect when they detect that probability of being
punished is below 30%.
26. Experimental design
• 100 Agents.
• Fully Connected Network.
• Results averaged on 25 runs.
• Different punishment distributions:
• Constant: agents are assigned a constant probability to punish.
• Linearly Increasing: probability continuously increases as the simulation
runs.
• Linearly Decreasing: the probability continuously decreases as the
simulation runs.
• Step Down: at a certain moment of the simulation, the probability of being
punished drops from 1 to 0, i.e. from total punishments to no punishment.
• Step Up: at a certain moment of the simulation, the probability of being
punished raises from 0 to 1, i.e. from no punishment to total punishments.
29. Experimental lessons
• Allowing different types of normative agents to interact
provides system policymakers with a tool that can help
predict the dynamics of prosocial behavior.
• Internalizers are endowed with a rich cognitive
architecture allowing high cooperation rates even when
punishment is low (and costly).
• Internalizers need a certain amount of strategic or
normative agents to dismiss the normative action when
necessary.
30. Conclusions
Summary
Building blocks of a rich cognitive
model of internalization
Types and degrees of internalization
factors favoring different types of
internalization
First implementation (EMIL-IA)
First experimental results on effect of
internalizaiton on cooperation rates
But “Only the barest outline of
internalization is known" (Vygotzsky,
1978, p. 57).
31. Conclusions (2)
Norme emerge from agents endowed with mental representations
and the capacity to act on them
A simpler agent architecture does not allow for norms to emerge
in certain domains
But to make norms happy, cognitive but non-fully deliberative
agent architecture is needed
Further artificial experiments,
more than one norm
more interesting scenarios
Necessity of cross-methodological experiments
Implement incorporation
Implement emotions