2. Overview
• Introduction
• The mature
neurophysiology of
biological motion (BM)
processing
• BM processing in typical
development and in ASD
• BM and social cognition
3. Biological motion and social cognition
• Social cognition - the
encoding and processing
in the brain of social
information.
• Biological motion (BM) is
a prime example of a
visual stimulus from
which the brain extracts
socially salient
information.
5. Biological motion and social cognition
• Intact BM processing is
essential for healthy
social interactions.
• Complex social
networks rely heavily on
BM information to
accomplish social tasks
across the brain.
6. Mirror neuron system
• “Unlike inanimate objects,
humans have the distinct
property of being ‘like me’ in
the eyes of the observer.
• This allows us to use the same
systems that process
knowledge about self-
performed actions, self-
conceived thoughts, and self-
experienced emotions to
understand actions, thoughts,
and emotions in others.”
Obermann and Ramachandran, 2007
7. Q: How is social information extracted
from sensory data?
10. Point-light displays
• Humans are capable of detecting socially
salient signals from extremely impoverished
sensory information.
• For example, we can detect diverse social
information from so-called biological motion
point light displays e.g. emotion, mood,
gender, and body-type.
14. Biological motion
• Biological motion (BM) detection is innate to
the healthy human brain.
• BM processing involves multiple sites across
the brain.
• The underlying neurophysiology remains
largely unexplored.
15. When and where is biological motion
processed in the mature brain?
Is it a simple, involuntary process?
17. Biological motion and attention
• Paradigm 1: Naïve subjects respond to a
distractor target (dot-color) enabling the
monitoring of the automatic processing of
unattended BM.
• Paradigm 2: Subjects are explicitly informed of
the presence of BM, and respond accordingly,
enabling the monitoring of attentional
processes.
18. Biological motion and attention
• High-density EEG
enables a thorough
mapping of bilateral
visual-social
pathways.
• Source localization
algorithms enable
the estimation of
neuronal generators
at different temporal
stages of the process.
Hans Berger (1873-1941)
19. Initial paradigm
Sample stimuli: on the left are still-
frames depicting normal biological
activity in point-light animation
sequences. On the right are the
scrambled counterparts of the
biological motion sequences.
20. unattended
attended
VEPs for the unattended (a)
and attended (b) biological
motion (BM) tasks. Blue lines
indicate the response to BM
stimuli, red lines indicate the
response to scrambled stimuli,
and green lines represent the
difference waves.
21. Scalp maps
Posterior topographic scalp
maps of the response during
both experimental conditions
and the difference maps
between them at selected
time-points.
22. Statistical cluster plots
BM vs. SM unattended
attended
Color-plot of t-values for the differences
between canonical biological motion
point-light displays and their scrambled
counterparts in the unattended (a) and
attended (b) tasks.
23. Brain Electrical Source Analysis
• BESA models intracranial dipoles that produce
observed scalp waveforms, using iterative
adjustments to reduce the variance between
the solution and the observed data.
• The upper bound of the number of modeled
dipole sources was determined using an
unconstrained test dipole.
24. Brain Electrical Source Analysis
Scalp maps of the difference
between BM and SM responses
and the corresponding source
localizations.
A. Talairach: x = 35, y = -69, z = -
2; explained variance [EV] =
91%
B. x = ±40, y = -69, z = 13; EV =
80%
C. x = 23, y = -80, z = 20; EV =
81%
D. x = ±40, y = -65, z = 7; EV =
89%
E. x = -37, y = -76, z = 16; x = 32,
y = -77, z = 10; EV = 93%
25. Brain Electrical Source Analysis
Summary of findings in some recent neuroimaging studies as related to our
source localizations. See, e.g., [KO] Tyler et al, 2005; [hMT] Becker et al, 2008;
[pSTS] Kontaris et al, 2009. (KO=kinetic occipital area; hMT=human homolog of
monkey medial temporal area; pSTS=posterior superior temporal sulcus)
26. How do these neurophysiological
processes develop in the healthy
brain?
36. Visual roots to social
dysfunctions
Biological motion processing
in children with autism
spectrum disorders
37. Autism spectrum disorders
1. qualitative impairment in social
interaction
2. qualitative impairments in
communication
3. restricted repetitive and stereotyped
patterns of behavior, interests and
activities
38. Autism spectrum disorders
Select symptoms:
– Has heightened or low senses of sight, hearing, touch, smell, or taste
– Cannot start or maintain a social conversation
– Does not adjust gaze to look at objects that others are looking at
– Does not refer to self correctly (for example, says "you want water" when
the child means "I want water")
– Does not make friends
– Does not play interactive games
– May not respond to eye contact or smiles, or may avoid eye contact
– May treat others as if they are objects
– Prefers to spend time alone, rather than with others
– Shows a lack of empathy
– May withdraw from physical contact because it is overstimulating or
overwhelming
– Doesn't imitate the actions of others
43. • There is a controversy in the literature
regarding the presence of BM-processing
differences in ASD.
• By using high-density EEG, the spatiotemporal
pathways used by TD and ASD populations can
be mapped out, resolving this debate.
ASD and biological motion
53. Mind A
Interactive
medium
Mind B
1 0 1
0 0 1
1 1 0
Motor
output
Motor
output
Sensory
input
Sensory
input
1 0 1
0 0 1
1 1 0
1 0 1
0 0 1
1 1 0
A
A
B
B
A
B
B
A
B
A
A
B
Representation
of other’s
internal states
Representation
of other
Representation
of self’s internal
states
Representation
of self
Representation
of medium
including BM
Representation
of medium
including BM
54. External
interactive
environment
1 0 1
0 0 1
1 1 0
Motor
output
Sensory
input
1 0 1
0 0 1
1 1 0
A
A
B
B
A
B
Representation
of other’s
internal states
Registry of self’s
internal states
Representation
of external
environment
including BM
1
1
0
1
0
1
Representation
of self’s motor
impact on
environment
Representation
of other’s motor
impact on
environment
Mind A
55. Conclusions
• Mature BM processing:
– 3 phases
• Normal development:
– Effects become automatic with age
– No early phase
– No attentional effects
• ASD:
– Like younger less automatic?
60. • Gender effects
• Schizophrenia
– Pilot data
• The inversion effect
– background
• Further directions
• Theory and philosophy of mind
• Ncl Closure effects and “objectness”
• Even more pilot analyses
– Minimum norms
– More clusterplots
• etc.
83. Future directions
• Audiovisual eye gaze and autism (ala Klin et al, 2009)
• Sensorimotor integration and ToM as a multisensory
(proprioceptive) phenomenon
• Electrophysiology of mentalizing and affect in ASD
and/or schizophrenia
• Ontological social processing…
• Closure and mismatch as neurocognitive currencies
• (way) future research – social-smiling infants: mindless
reflex or social reward/ToM
• Audiovisual interpolated and full bodies and faces vs.
coherent/meaningful/inanimate motions
84. Low level sensory processes vs. higher
order social cognition
• VEPs and visual field confounds
– Biological motion vs. scrambled and inverted
displays
– Scrambled “biological motion” vs. non-biological
coherent motion (and Aristotelian spheres/
motion and mind…)
– Online “minded” event and artifacts… (random
structures vs. teleological structures
(tools/art/etc.)
– Affect and ASD
92. Social class formation
• “Properties” and “objects”
• Social construct formation
• Theory of mind
93. Abstract vs. concrete perception
Necessary / abstract
• Conceptual
• Mathematical / formal
science
• Possibility super-space/set
• Denominator in
probabilistic reasoning /
stats / signal detection
• E.g.:
– 1+1=2
– p^~p=F
Actual / concrete
• Sensory
• Selection
• Subspace
• Possibility subspace
• Numerator in prob.
Reasoning
• E.g.:
– The sky is blue.
– It weighs 13 lbs.
94. Semantic word classes / percepts
social
• I
• You
• He
• She
• Person
• Mind
• Consciousness
• Thought
• See
• Hear
• Feel
• Sense
• Emotion
• Want
• Desire
• Love
• Hate
• Guilt
concrete
• It
• Inanimate
• Object
• Matter
• Energy
• Space
• Time
• Function
• Equation
• Law
• Event
• Particle
• Motion
• Position
• Size
• Acceleration
• Color
• Temperature
• Sound
106. What is “social cognition”?
• Environmental processing
– Sensory data (mismatches)
– Data symmetries (closures)
– Signal – noise inductions
(closure effects)
• Sense of self
– Bayesian input and output
– (sense of possible and
actual)
• Sense of other
– Social signal detection
201. Mirror neuron system
• “Unlike inanimate objects,
humans have the distinct
property of being “like me” in
the eyes of the observer.
• This allows us to use the same
systems that process
knowledge about self-
performed actions, self-
conceived thoughts, and self-
experienced emotions to
understand actions, thoughts,
and emotions in others.”
Obermann and Ramachandran, 2007
202. What’s new with mu?
• Mu wave suppression
and mirror neurons
• Implicated in ASD?
– Raymaekers et al., 2009
• No difference in ASD in
contrast to other studies?
203. Beyond mirror neurons
• 1st-person (empathetic) vs 3rd-
person (systematic)
processing model of
neurocognition
– Animacy scales
– “selfness” vs. “otherness”
• Extreme male brain and higher
empathy/identification
threshold
– Reductionism: Selections and
selectors
• Cognition of the possible
• Cognition of the actual
• Non-determinism
Notes de l'éditeur
We then conducted pointwise, paired, two-tailed t-tests between the VEP responses.
Not seeded
Brain electrical source analysis
BESA models intracranial dipoles that produce observed scalp waveforms, using iterative adjustments to reduce the variance between the solution and the observed data.
The upper bound of the number of modeled dipole sources was determined using an unconstrained test dipole.
a. Scalp map of the difference between BM and SM responses at ~140 ms in the unattended task and the corresponding source localization for the 120-160 ms time-window (Talairach: x = 35, y = -69, z = -2; explained variance [EV] = 91%). b. Scalp map of the difference between BM and SM responses at ~275 ms in the unattended task and the corresponding symmetric sources localized for the 200-350 ms time-window (Talairach: x = ±40, y = -69, z = 13; EV = 80%). c. Scalp map at ~140 ms of difference-waves between scrambled and canonical biological motion for the attended task and the corresponding source localized for the 120-160 ms time-window (Talairach: x = 23, y = -80, z = 20; EV = 81%). d. Scalp map at ~275 ms of the difference between scrambled and canonical biological motion for the attended task and the corresponding symmetric sources localized for the 200-350 ms time-window (Talairach: x = ±40, y = -65, z = 7; EV = 89%). e. Scalp map at ~450 ms of the difference between scrambled and canonical biological motion for the attended task and the corresponding sources localized for the 400-500 ms time-window (Talairach: x = -37, y = -76, z = 16; x = 32, y = -77, z = 10; EV = 93%).
Brain electrical source analysis
BESA models intracranial dipoles that produce observed scalp waveforms, using iterative adjustments to reduce the variance between the solution and the observed data.
The upper bound of the number of modeled dipole sources was determined using an unconstrained test dipole.
Systemizers vs. empathizers
Uta frith
Simon baron cohen
Despite similar behavioural performance, significantly less activity was found for the AS group (relative to a control group) in inferior, middle and superior temporal regions, including the human analogue of MT+/V5. These data suggest that AS is associated with unique patterns of brain activity during the perception of visually presented social cues. Areas of reduced brain activity among individuals with Asperger Syndrome during the randomised and nonrandomized walker conditions. The statistical maps represent the comparison of brain activity across the two groups for the contrast of non-randomised walker versus fixation (top) and randomised walker vs. fixation (bottom), overlayed on a template brain that has been normalised into the MNI coordinate systems and rendered using mri3dX (http://www.jiscmail.ac.uk/lists/mri3dx.html). Clusters are significant at p < .004 for the nonrandomized walker contrast, and .006 for the randomised walker contrast (each corresponding to a Type I error probability of less than one cluster). Activations in yellow represent clusters of activity that significantly lower for the AS group as compared to controls.
Faces with a normal (intact) broad spatial frequency (BSF) content (left column) were filtered to contain only a high range or low range of spatial frequencies (HSF or SLF; middle and right columns respectively). Six possible face types were equally distributed in a 2 3 factorial design (n = 40 per cell), with either a fearful or neutral expression (half each), and either a BSF, LSF or HSF content (all shown in random order; duration 200 ms each; mean interstimulus interval 6.1 s). Individual face stimuli were counterbalanced across participants, with different expressions and spatial frequency contents chosen for each face across participants. In addition, each individual was repeated once (after a randomly chosen lag of between 25 and 35 intervening stimuli) with either the same or a different spatial frequency content (that is, same or different image of the same person; n = 20 faces for each type of repetition; see Results).
High and low spatial frequency information in visual images is processed by distinct neural channels. Using event-related functional magnetic resonance imaging (fMRI) in humans, we show dissociable roles of such visual channels for processing faces and emotional fearful expressions. Neural responses in fusiform cortex, and effects of repeating the same face identity upon fusiform activity, were greater with intact or high-spatial-frequency face stimuli than with low-frequency faces, regardless of emotional expression. In contrast, amygdala responses to fearful expressions were greater for intact or low-frequency faces than for high-frequency faces. An activation of pulvinar and superior colliculus by fearful expressions occurred specifically with low-frequency faces, suggesting that these subcortical pathways may provide coarse fear-related inputs to the amygdala.
History of motion capture - late 1800s - scientist Étienne-Jules Marey and Eadweard Muybridge conducted independent studies of human and animal motion. used multiple photographs of moving objects to present the biological movement (Menache, 2000). major impact in biology, medicine, photography and nowadays also in animation (Menache, 2000).
VEPs for the unattended BM task. Blue lines indicate the response to BM stimuli, red lines the response to scrambled stimuli, and green lines represent the difference waves.
VEPs for the attended BM task. Blue lines indicate the response to BM stimuli, red lines the response to scrambled stimuli, and green lines represent the difference waves.
Stimulus conditions. A, Experiment 1: intact unoccluded, intact occluded, apart
unoccluded, and apart occluded stimuli. B, Experiment 2: upright unoccluded, upright occluded,
inverted unoccluded, and inverted occluded stimuli. C, Examples of Y (anteroposterior)
and Z (inferior–superior) rotation of one gait from two parent segments and their children as a
function of video frame (parent, left arm; children, left forearm, left hand; parent, right leg;
children, right thigh, right shin). Note that the same Y, Z (and X, not shown) rotation for each
segment was used in the intact and apart conditions. D, Cross-correlations of Y and Z rotation
from the left hand and left forearm (blue), right thigh (red), and right shin (green) Y rotation as
a function of frame. The cross-correlation between Y (or X or Z) rotations of different segments
was identical across intact and apart conditions. Expt, Experiment.
Overlap in effects between experiments 1 and 2. Group analysis activation data
overlaid on a single subject’s normalized SPGR images (top) and inflated Colin brain atlas anatomical
images of the left (bottom left) and right (bottom right) lateral hemispheres. Green
indicates significant activation to all experimental conditions from experiments 1 or 2 versus
fixation. Red indicates a significantly greater response to intact versus apart, regardless of
occlusion from experiment 1. Yellow indicates interaction between intactness and occlusion,
from experiment 1. Purple indicates a significantly greater response to upright versus inverted,
regardless of occlusion, from experiment 2. Symbols indicate location of peak activation in EBA
in previous studies. STG, Superior temporal gyrus; pITS, inferior temporal sulcus, posterior segment;
L, left; R, right.
Experiments (Expts) 1 and 2: mean1 SE percentageMRsignal change (% signal
change) relative to fixation baseline at maximally responding anatomical locations (with corresponding
centroids in Talairach coordinates). Data from experiments 1 and 2 are presented in
left and right columns, respectively. White bars represent intact (Expt 1) or upright (Expt 2)
conditions, gray-filled bars represent apart (Expt 1) or inverted (Expt 2) conditions. The activations
for the right STS and right ITS were presented originally in Figure 2, left POF in Figure 3, and
right STS/STG and right pITS in Figure 4. Unoccl, Unoccluded; Occl, occluded; POF, parietooccipital
fissure; STG, superior temporal gyrus; pITS, inferior temporal sulcus, posterior
segment.
The averaged amplitude and latency of the P1 component. (A) O1/O2 electrodes and (B) T5=/T6= electrodes. The error bars indicate the
standard errors (SE) of the mean.
The averaged amplitude and latency of the N1 component. (A) O1/O2 electrodes and (B) T5=/T6= electrodes. The error bars indicate the
standard errors (SE) of the mean.
The averaged P1-N1 amplitude. (A) O1/O2 electrodes and (B) T5=/T6= electrodes. The error bars indicate the standard error (SE) of the
mean.
The averaged amplitude and latency of the N2 component at the T5=/T6= electrodes. The error bars indicate the standard errors (SE) of the
mean.
Regression analysis of the P1 amplitude (left panel) and N1 latency (right panel) at the (A) O1/O2 and (B) T5=/T6= electrodes. We found a
significant negative correlation except for the N1 latency of the sPLW stimulus at T5= electrode (see Table 3).