Despite overwhelming evidence that many aspects of human cognition are highly context-dependent, generic (that is models that are supposed to hold across different contexts) abound, including: most models of rationality and decision making, and most models that are based on statistically fitting equations to data. Context itself, especially social context, has been systematically by-passed by both quantitative and qualitative researchers. Quantitative researchers claim to be only interested in those patterns that are cross-context. Qualitative researchers only deal with accounts within context. Neither tackle the nature of context itself: how it works, in what ways it impacts upon behaviour.
Dealing with context is notoriously hard: the concept is slippery and its effects hard to identify. However, I claim it is not impossible to research. A combination of rich datasets and newer computational methods could help (a) identify some social contexts and (b) relate what happens within a context to how contexts are collectively constructed. Such a step could help relate quantitative and qualitative evidence in a way that is better founded and hence, perhaps, open the way to the unification of the social sciences as a coherent discipline.
The Scandal of Generic Models in the Social Sciences
1. The Scandal of
Generic Models in the Social Science
Bruce Edmonds
Centre for Policy Modelling,
Manchester Metropolitan University
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 1
2. Outline of Talk
1. The Main Point
2. A Bit About Context
3. Social Roots of Cognitive Context
4. Defensive Responses
5. Some Ways Forward
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 2
3. Part 1:
The Main Point
(in four slides)
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 3
4. The Elephant in the Room
• Many aspects of human cognition are known
to be highly context-sensitive, including:
memory, preferences, language, visual
perception, reasoning and emotion
• There is a mountain of qualitative research that
has documented instances where a specific
context is essential to understanding the
observed behaviour
• Simple observation and introspection tells us
that behaviour in different kinds of situation is
not only different but decided on in different
ways (e.g. in a lecture and a football game)
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 4
5. However despite this…
• Almost all formal models of human
behaviour (mathematical, logical or
computational) are generic – they do not
exhibit this sharp context-dependency
• Another stream of models (models fitted to
or tested against data) consider a single
model (at a time) against a set of data that
derives from many different contexts
• This seems to me to be a case of massive
“wishful thinking”
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 5
6. Qualitative research…
• Does take context seriously in a way, but has
(largely) retreated to description within specific (and
described) contexts
• Knowledge is only useful if it is to some extent
applicable in a new situation (even if only slightly
new)
• Analogical reasoning can use knowledge from one
context to be projected upon another, and this can
give insights (interesting hypotheses) but not
reliable knowledge
• This kind of research often avoids responsibility for
the application of knowlnecessarily in a different
situation to where it was observededge gained from
its studies () whilst implying it is somehow useful
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 6
7. Context
To summarise:
• We have one set of researchers who are ignoring
context, optimistically hoping to find general
patterns even though they must know context must
be crucial in many cases
• Another set of researchers refuse to look at what is
general across contexts or how contexts might
systematically affect behaviour
• Not many are seriously trying to study social context
itself – how it works, what regularities there are,
how to identify it, how to model its impact
• Social context is central to human behaviour but
effectively not researched much
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 7
8. The Point of the Talk is…
1. To look at the role context (again!)
2. To look at (and argue against) some of
the arguments that are raised to justify
ignoring context
3. To persuade you to take context
seriously, not just as an “add-on”
4. To suggest some positive ways forward
5. To (hopefully!) motivate you to join this
project and help contribute methods
that will help us seriously tackle context
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 8
9. Part 2:
A Bit About Context
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 9
10. The Difficulty of Talking about
Context
• The word “context” is used in many different
senses across different fields
• Somewhat of a “dustbin” concept resorted to
when more immediate explanations fail (like
the other “c-word”, complexity)
• Problematic to talk about, as it is not clear that
“contexts” are usually identifiably distinct
• Mentioning “context” is often a signal for a
more “humanities oriented” or
“participatory/involved” approach and hence
resisted by “scientists” who are seeking
general laws
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 10
11. A (simplistic) illustration of context from the
point of view of an actor
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 11
12. Situational Context
• The situation in which an event takes place
• This is indefinitely extensive, it could include
anything relevant or coincident
• The time and place specify it, but relevant
details might not be retrievable from this
• It is almost universal to abstract to what is
relevant about these to a recognised type
when communicating about this
• Thus the question “What was the context?”
often effectively means “What about the
situation do I need to know to understand?
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 12
13. Cognitive Context (CC)
• Many aspects of human cognition are context-
dependent, including: memory, visual perception,
choice making, reasoning, emotion, and language
• The brain somehow deals with situational context
effectively, abstracting kinds of situations so
relevant information can be easily and preferentially
accessed
• The relevant correlate of the situational context will
be called the cognitive context
• It is not known how the brain does this, and
probably does this in a rich and complex way that
might prevent easy labeling/reification of contexts
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 13
14. The Context Heuristic
• The kind of situation is recognised in a rich,
fuzzy, complex and unconscious manner
• Knowledge, habits, norms etc. are learnt for
that kind of situation and are retrieved for it
• Reasoning, learning, interaction happens
with respect to the recognised kind of
situation
• Context allows for the world to be dealt with
by type of situation, and hence makes
reasoning/learning etc. feasible
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 14
15. Implications of the Context Heuristic
• One would expect whole clusters of norms,
habits, expectations, knowledge, language,
ways of making decisions etc. etc. to be
associated with a context
• Although we may use various proxies for
context (place in network, physical location,
occurrence of co-occurring words etc.) kinds of
situation are recognised in an information rich
manner, often with quite subtle social signals
• It is a fallible heuristic…
• …so why do we have this kind of cognition?
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 15
16. Part 3:
Social Roots of Cognitive Context
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 16
17. Social Intelligence Hypothesis
• Kummer, H., Daston, L., Gigerenzer, G. and Silk, J. (1997)
• The crucial evolutionary advantages that
human intelligence gives are due to the
social abilities it allows
• Explains specific abilities such as imitation,
language, social norm instinct, lying,
alliances, gossip, politics etc.
• Social intelligence is not a result of general
intelligence, but at the core of human
intelligence, “general” intelligence is a side-
effect of social intelligence
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 17
18. An Evolutionary Perspective
Social intelligence implies that:
• Groups of humans can develop their own
(sub)cultures of technologies, etc. (Boyd and
Richerson 1985)
• These allow the group with their culture to
inhabit a variety of ecological niches (e.g.
the Kalahari, Polynesia) (Reader 1980)
• Thus humans, as a species, are able to
survive catastrophes that effect different
niches in different ways (specialisation)
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 18
19. Implications of SIH
• That different complex “cultures” of knowledge
are significant
• An important part of those cultures is how to
socially organise, behave, coordinate etc.
• One should expect different sets of social
knowledge for different groups of people
• That these might not only be different in terms
of content but imply different ways of
coordinating, negotiating, cooperating etc.
• That these will relate as a complete “package”
to a significant extent
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 19
20. Social Embedding
• Granovetter (1985)
• Contrasts with the under- and over-socialised
models of behaviour
• That the particular patterns of social
interactions between individuals matter
• In other words, only looking at individual
behaviour or aggregate behaviour misses
crucial aspects
• That the causes of behaviour might be spread
throughout a society – “causal spread”
• Shown clearly in some simulation models
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 20
21. Implications of Social Embedding
• In many circumstances agents can learn to
exploit the computation and knowledge in their
society, rather than do it themselves (invest in
what Warren Buffet invests in)
• Knowledge is often not explicit but is
something learned – this takes time
• This is particularly true of social knowledge –
studying guides as to living in a culture are not
the same as living there for a time
• Social embedding means that human
behaviour can not be understood well separate
from its cultural context
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 21
22. The Social Co-Development of Shared
Recognised Context
• Over time, due to their similarities, certain kinds of
situation become recognised as similar by
participants
• This facilitates the development of a set of shared
habits, norms, knowledge, language etc. that is
specific to the context
• The more this happens the more distinctive that
kind of situation becomes and hence more
recognisable by newcomers
• Eventually these may become institutionalised in
terms of infranstructure, training etc. (e.g. how to
behave in a lecture theatre)
• This co-development of context may be the reason
for its social/evolutionary value
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 22
23. Implications of the Social Roots of
Context-Dependency
• Behaviour of observed actors might change sharply
across different social contexts
• The relevant behaviour, norms, kinds of interaction
etc. might also change
• Social contexts are co-developed and changing
• They may be different for different groups
• Some kinds of social behaviour seem to be
inherently context-dependent (compliance)
• It is unlikely that a lot of key social knowledge,
norms, behaviour etc. will be generic
• Models that assume a cross-context engine of
human behaviour may be deeply misleading!
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 23
24. Part 3:
Some Defensive Responses
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 24
25. Some Possible Responses
• Its too difficult, I’ll continue to ignore it
• I am only looking at the wider/more general picture,
what is common across contexts
• I treat intra-context variation as random noise
• I have included context, it is the variables/features
a, b, c etc. which vary with the context
• I am acting within context only
• I am only modelling a single context
• It is not scientific
• We can achieve generality through simpler models
• I need an analytic expression for my model
• Use natural language/analogical models only
• I don’t have enough data
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 25
26. Background – wishful thinking and
the resistance to change
• Everyone (me included) has developed certain
core habits/skills and is loth to abandon these,
since it is a lot of effort
• There is also a tendency to shape our
opinions/arguments to be coherent with our
goals (rather than the other way around)
• We should not assume that the world we study
(the social world) is organised for our benefit
(as academics studying it)
• Thus I ask you to resist the impulse to react
against “yet another difficulty” – the difficulty is
real and exists even if we ignore it
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 26
27. Ignoring Context
• Much modelling happens with a single
context in mind, in which case it can be
ignored but only if
– everyone is using the same idea of this context
– there is no significant “leakage” of causation
from outside the background, that is the scope
is wide enough to include all significant
influencing factors
– The actors/organisms don’t deal with the same
situation as different cognitive contexts
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 27
28. The “Simple is more General” Fallacy
• If one has a general model one can make it
more specific (less general) by adding more
processes/aspects…
• …in which case it can become more complex
• However, the reverse is not true…
• If one simplifies/abstracts then you don’t get a
more general model (well almost never)!
– there may be no simpler model that is good
enough for your purpose
– But, even if there is, you don’t know which aspects
can be safely omitted – if you remove an essential
aspect if will be wrong everywhere (no generality)
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 28
29. Context-Dependency and
Randomness
What
appears to
be random
may in fact
be due to
variation of
context
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 29
30. Using Randomness as a Proxy for
Contextual Variation
• It is completely unsurprising that many factors
will be significantly correlated with many
outcomes in a multi-context situation
• It is also unsurprising that the explanatory level
of such exercises are unimpressive
• The correlation may be due to a strong
correlation in only one kind of context and,
indeed, mask anti-correlations in others
• Using randomness as a proxy for contextual
variation simply discards a lot of the
information in the phenomena – it amounts to
ignoring evidence
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 30
31. Scaling by Size
• Look at variance as system size increases…
• Does variance as a proportion of size disappear?
• In this case Law of large numbers does not apply
• Simple examples:
• Kaneko (1990): parallel globally coupled chaotic processes
• Edmonds (199?): scaling Brian Arthur’s “El Farol Bar” Model
Contextual variation
Variance
(scaled by size)
Model with random noise
Size
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 31
32. Context-Dependency
and “Being Scientific”
• If the relevant context can be reliably identified
then…
• …context-dependency is not the same as
subjectivity (even if there are a some hard
cases that escape definition)
• Generality is nice if you can get it, but its no
good pretending to have it if you can’t
• Science should adapt to what it wishes to
understand, not the other way around
• It does mean (often) an acceptance that
general/generic approaches are not useful
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 32
33. Analogical Thinking
• Humans are good at using analogies, relating an idea or
example from one context to another in a rich, relevant
and flexible manner – it is a powerful method of thought
• They build the mapping from the analogy to the a
context “on the fly”, largely unconsciously
• The mappings are different each time an analogy is
applied, thus not a reliable source of transmittable
knowledge – each person might build a different
mapping unless they inhabit the same context
• Many published models do not have an explicit mapping
to a domain, but are used more as analogy
• This is sometimes hidden, so when a simulation (or
analytic model) does not directly map to observations
but to an idea which then applies as an analogy to the
domain and not directly, this gives a spurious
impression of generality
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 33
34. Part 4:
Some Ways Forward
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 34
35. Some ways forward
• Keeping the data and simply NOT summarising it (at
least not prematurely)
• Data mining local patterns to detect commonality of
multiple models/measurements across similar contexts
• More complex simulation models with context-
dependent cognitive models
• Context-sensitive microsimulation models
• Context-oriented visualisation techniques
• Integrating personal/anecdotal accounts of behaviour –
making use of qualitative evidence with its context
• Not leaving the context(s) – acting within the normal
sphere of shared and relevant situations
• Staging abstraction more gradually
• Clusters of related models covering different contexts
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 35
36. Data-mining Rich Data
• Rich Data is data that:
– Encodes many different aspects of the
event/situation/actor
– Is over time (not one-shot) and reasonably
frequent
– Is not merely lots of data if this is thin (e.g.
second-by-second coordinates of where a
person is)
• The first stage is to look for “Local Patterns”
– patterns that hold for a part of the data
along with their scope
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 36
37. Cleveland Heart Disease Data Set – the
processed sub-set used
In processed sub-set:
• 281 entries
• 14 numeric or numerically coded attributes
• Attribute 14 is the outcome (0, 1, 2, 3, 4)
• Some attributes: age, sex, resting blood
pressure (trestpbs), cholesterol (chol),
fasting blood sugar (fbs), maximum heart
rate (thalach), number of major vessels (0-
3) colored by flourosopy (ca)
• From the Machine Learning Repository
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 37
38. Fitting a Global Model (R=56%)
Num = -0.01*age + 0.17*sex + 0.20*cp + 0.00*trestbps + 0.10*restecg + -
0.01*thalach + 0.23*exang + 0.18*oldpeak + 0.16*slope + 0.43*ca + 0.14*thal + -
0.60 (+/- 0.83)
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 38
39. Looking for Clusters in HD Data Set
(Start of Process)
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 39
40. Final Set of Clustered Solutions
• Final solution
set after some
time.
• Still complex but
some structure
is revealed
• Note presence
of “fbs” despite
not being
globally
correlated and
that “chol”
helped define
the context
space
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 40
41. Clusters of Model Scopes may
suggest a ContextM 1
M1 M2
suggests a context
A useful context is one that:
– includes related models with different
goals/predictions but similar scope
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 41
42. Need for a Meta-Clustering
Algorithm
Suggests
Time of Day a Context
Item Type Shop
Location
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 42
43. Adding Context-Dependency into
Simulation Models
• Wander has argued for a needs-oriented
cognitive architecture for the agents in a
simulation
• Here the idea is to allow the agents in a
simulation to learn/reason/decide in a
context dependent fashion
• This does not affect the method of each of
these, but does change what knowledge,
expectations etc. the methods act upon
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 43
44. Basic Cognitive Model
Reasoning/plan
Context
ning/belief
Recognition
revision/etc.
Context-Structured
Memory
• Rich, automatic, imprecise, messy cognitive
context recognition using many inputs
(including maybe internal ones)
• Crisp, costly, conscious, explicit cognitive
processes using material indicated by
cognitive context
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 44
45. Example: an artificial stock market
• Traders decide which of several stocks to hold,
or to keep cash
• Fundamentals are the dividend rates which
follow slow random walks
• Agents evolve and evaluate possible strategies
by what their wealth would be (including all
aspects like transaction costs) of using these
over a past time period
• Some agents can be given a context-
dependent memory of strategies, others a
universal store of strategies
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 45
46. Example – models in the
cognition of a trading agent
950
Volatility - past 5 periods
900
850
800
750
700
750 850 950
Volume - past 5 periods
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 46
47. The model contents in snapshot of
one trader
model-256 priceLastWeek [stock-4]
model-274 priceLastWeek [stock-5]
model-271 doneByLast [normTrader-5] [stock-4]
model-273 IDidLastTime [stock-2]
model-276 IDidLastTime [stock-5]
minus
[divide
[priceLastWeek [stock-2]]
model-399 [priceLastWeek [stock-5]]]
[times
[priceLastWeek [stock-4]]
[priceNow [stock-5]]]
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 47
48. Total Assets in a Typical Run
30000
Total Value of Assets
25000
20000
15000
10000
5000
0
0 100 200 300 400 500
Time
Black=context, White= non-context
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 48
49. Lessons from Stock Market Example
• Traders with a context-dependent memory
learned to recognise certain market conditions
and develop/select strategies specifically for
these
• Though the context heuristic was not always
better – e.g. if the market was very turbulent
then it is better to act fast and adaptively
• In any case “context traders” acted in a
different kind of pattern to “generic traders”
• The presence of “context traders” changed
how the whole market reacted
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 49
50. Integrating Aspects of Qualitative
Evidence into Formal Models
• Identifying kinds of context (those over which we
might expect some regularity in terms of shared
norms, expectations etc.) might allow suggestions
from qualitative evidence to be incorporated into
agent-based models
• For example by providing the repertoire of possible
strategies in the context which are decided between
• This could greatly enrich agent-based models
allowing in some of the social “mess” we observe
• However this requires new methods to analyse
narrative evidence in a context-depenent manner
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 50
51. Analysing Narrative Evidence in a
Context-Sensitive Manner
• It is hard since people assume context, it is
usually left implicit, indeed people are often not
aware of the context they are assuming
• However…
– Socially entrenched contexts can be identified
– When giving examples (could you imagine doing
that in situation X) people are relatively good at
recognising when the context is wrong
– Our intuitions are a good starting point, as long as
we are aware they might be wrong
• No well-developed methods – this needs
further research
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 51
52. Conclusions – Prospects for
Researching Context
• Ignoring it and simply hoping it won’t matter is not
an option (if we are serious about our project)
• There are ways forward to meaningfully make
progress in dealing with context-dependency
• And some of these involve the integration of
qualitative/in situ approaches with
quantitative/formal modelling
• We will need a LOT more data both
multi-dimensional and at a finer-granularity, but this
is starting to come on stream
• Ignoring context seems to be impeding the
integration of both: action-oriented and model-
based approaches, as well as quantitative and
qualitative approaches
• Please help
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 52
53. Acknowledgements
Many thanks to all those with whom I have
discussed these ideas, including…
Emma Norling, Nick Shryane, Jason Dykes,
Scott Moss, Wander Yager, Cathy Urquhart;
those at the Conference Series on “Modelling &
Using Context”;
the regulars at the Manchester Complexity Seminar;
those in the SCID Project;
and the participants of the recent Lorentz workshop
on “Formal Methods for an Informal World”.
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 53
54. The End
Bruce Edmonds
http://bruce.edmonds.name
Centre for Policy Modelling
http://cfpm.org
The Scandal of Generic Models in the Social Sciences, Bruce Edmonds, Gronigen, , April 2013, slide 54
Notes de l'éditeur
AI, NL, Sociology, Philosophy, Mobile devices, Psychology, Cognitive ScienceFor detailed argument seem my previous papers on thisDustbin Like complexitywill talk about this problem later
Imagine a professor of physics in a wild place – does his intelligence help him to survive?
Reader 1980, Man on Earth
leakage noisenot the case where un-modelled aspects are effectively randomdiscuss random gas example