1. Valuation and choice: Using values
PSYC 596F
10/20/2009
Presenter: David Yokum
Chapter 28: Antonio Rangel. The computation and
comparison of value in goal-directed choice.
Chapter 29: Michael Platt and Camillo Padoa-Schioppa.
Neuronal representations of value.

2. Outline
• Rangel: How does the brain compute and
compare values? The random-walk model.
• Platt and Padoa-Schioppa: What are the
neural substrates of value computations?
OFC, LIP, and CGp.
• Throughout, your questions and comments!

3. Value-based decision making
1) Create representation of decision problem
2) Assign values to considered actions
3) Compare valuations
(select highest valued action)
4) Measure desirability of outcomes
5) Use feedback to update

4. Simplifications
1) Create representation of decision problem
2) Assign values to considered actions
3) Compare valuations
(select highest valued action)
4) Measure desirability of outcomes
5) Use feedback to update
 Goal-directed choice:
– two options
– familiar outcomes
– immediate consumption

5. Random dot motion task
Fig. 28.1

6. Random-walk model (RWM): measure
of relative evidence
1. Encode moment-to-moment sensory evidence
2. Accumulator measuring net evidence
3. Thresholds implementing decision criterion
Fig. 28.2

7. RWM: behavioral evidence
Fig. 28.3

8. RWM: neural correlates (1)
• Middle temporal area contains groups of
neurons that respond preferentially to stimuli
moving in certain directions (i.e., response
fields); activity increases with coherence.
Fig. 28.5

9. RWM: neural correlates (2)
• Time-courses of neural activity in lateral
intraparietal area rise to single threshold
about 100 ms before saccade; an accumulator
signal?
Fig. 28.6

10. RWM: neural correlates (3)
• What about threshold computation?
• Maybe LIP? Maybe superior colliculus?

11. Simple goal-directed choice
Fig. 28.7

12. RWM redux: attention-driven choice
• Relative value again computed, but this time
an advantage is given to whatever item is
being attended to.
Fig. 28.8

13. AD-RWM: behavioral evidence
Fig. 28.9

14. Random walk model: Evaluation?
• Filippo: Rangel is worried that the model is
limited to binary choice, but this concern is
unwarranted.
• How is threshold determined?

15. Neural substrates of value
computations
• Orbitofrontal cortex (OFC): absolute value of
good
• Lateral intraparietal parietal cortex (LIP):
relative value of good
• Posterior cingulate cortex (CGp): subjective
value; update LIP

16. Economic choice task
Fig. 29.1

17. Economic choice task
Fig. 29.1

18. OFC and chosen value (i.e., value of
whatever juice ultimately chosen)
Fig. 29.2

19. OFC and offer value (i.e., value of one
of the two juices alone)
Fig. 29.3

20. OFC and taste
Fig. 29.3

21. OFC invariant for menu changes
Fig. 29.4

22. OFC upshot
• Representation of value in OFC is independent
of visuo-motor contingencies and menu
invariant.
• Essentially a cardinal, or absolute,
representation of value.

23. OFC data: Evaluation?
• Trevor: “I don't know much about how dementia
works. When dementia patients have OFC
damage, is it selective to a particular part of the
OFC, or to neurons in the OFC that either get
their inputs from or project to specific regions, or
is it just widespread, unselective damage to the
OFC? If it is selective, is it possible that the cell
death corresponds to neurons of particular
functional types (chosen, offer, or taste)
neurons?”

24. OFC data: Evaluation?
• Trevor: “The authors present at least 4
different functional types [of info encoded by
OFC neurons]: chosen value, offer value A,
offer value B, and taste. Do we know anything
about the properties of these different
neurons? Are "offer" neurons and "chosen"
neurons different in any meaningful ways or
part of different functional networks in the
brain?”

25. OFC data: Cardinal versus ordinal
• Filippo: Problems with conclusion that OFC activity
represents cardinal value. Opposing data from Tremblay
and Schultz (1999).
“[M.P. and C.P-S.] briefly discuss the clash between the two lines of research by
noting that i) they may have recorded from different orbital areas and ii) that the
paradigm adopted in Tremblay et al. may have induced adaptation, which in turn
may explain the differences between the two sets of results. They then conclude
this brief line of reasoning by claiming that the “cardinal hypothesis” is confirmed
by empirical data.
“Apart from the fact that the strength of the argument is questionable, it is also
not clear how one can discern encoding of ‘absolute value’ from other possible
information…”
• Luke: “…the conclusion that it [OFC] calculates "objective"
value seems overstated and tenuous at best.”

26. Lateral intraparietal cortex (LIP)
• Linking sensory signals with motor commands,
as well as guiding sensory attention.
• Modulated by expected value of different
stimuli?
– Enter Michael Platt’s c.v.
– LIP neuronal activity proportional to EV of target
(e.g., altered juice amount or probability of cue)

27. LIP: pay-per-view task
Fig. 29.5

28. Fig. 29.6
LIP: abstract, relative value

29. LIP signal independent of modality
• “These findings suggest that decisions based
on value operate on a common currency that
is independent of the modality of the goods
under consideration or the actions they
motivate” (450).

30. Question
• Phil: “Is there a difference between "primary" rewards
(such as food/drink), "secondary" rewards (such as
money), and "social" rewards (such as viewing pictures
of attractive members of the opposite sex): Clearly the
value of food/drink varies with how hungry the animal
(rat/monkey/human) is, while a secondary reward has
value only in that it can be exchanged for a primary
reward (i.e. Monopoly money is fairly useless), but
social rewards might be different - for instance, can
one ever get bored of looking at photos of pretty
women? If hunger/thirst mediate the value of
food/drink, what might mediate the value of photos of
attractive women?...”

31. Question
• Trevor: “I found the brief discussion of
mutlimodality valuation very interesting and
wanted more information about studies in
humans that have tried to assess whether
valuation is done the same across different types
of information. Aside from the studies with
female and male images discussed in the chapter,
and some of the charitable giving research, are
there any good examples of studies that have
tried to look at decisions in which multiple
modalities must be considered when making a
choice?”

32. LIP data: Evaluation?
• Mirre: “Are LIP neurons only involved in
perceptual decision-making or has LIP activity
only been measured using saccadic decision-
making tasks? Do LIP neurons in human
beings function similarly as in monkeys?”

33. LIP data: Evaluation?
• Kaisa: “The fact that different economical variables are reflected in the LIP has
been puzzling me for quite a while. What does it really tell us in studies where
monkeys are answering with eye movements? Somehow I have gotten the
feeling that always when there is a significant difference in the behavior, LIP
neurons also show the same effect (sometimes amazingly accurately). Are the
measurements of LIP activity and behavioral responses (saccades) actually
indicating the same thing at different stages, i.e., selection of motor action and
the action itself? Is the choice calculated elsewhere and fed forward to LIP?
Could the monkeys still detect the direction of random dot motion if they would
not have means of responding at the time of motion display (no knowledge
which saccade to make)? If the monkeys could learn to use delayed response
instructions (appearing after the motion display disappeared), what kind of
activity would there be in LIP prior to the saccade (fast accumulation in every
condition)? Does anybody know if this kind of studies have been made?
And if monkeys are able to make 'abstract' choices about the direction of
motion, which areas are involved in the computations? There could be a region
coding for the choice irrespective of any spatial location, similarly to the OFC
coding value of goods.”

34. LIP data: Evaluation?
• Luke: “No disrespect to Michael Platt and colleagues, but I
found something lacking in their argument and overall
approach. For reasons that have never been clearly spelled
out, he has spent over a decade studying value in regions in
which there is no clear evidence (nor ever has been)
supporting their role in its calculation. At the conclusion of
the chapter he notes what has been painfully obvious to
everyone else for years - lesioning the LIP does not result in
an impaired ability to calculate value. However, it does impair
attention and ability to initiate visual saccades (a behavior
that is crucial for his tasks…”

35. “…Not determined to give up on studying
regions that have no clear role in calculating
value, he decides to focus on the posterior
cingulate gyrus…” - Luke

36. Posterior Cingulate Cortex (CGp)
• Anatomical and neurophysiological evidence
suggest a role in “signaling motivationally
significant events and actions, as well as
perhaps their subjective value for guiding
future behavior.”

37. CGp signals are subjective

38. CGp and evaluation of stimuli
• CGp neurons respond most strongly to visual
events that are unpredictable in space or time
(Dean et al., 2004)
• CGp neurons respond following delivery of
unpredictable reward or omission of
predictable reward (McCoy et al., 2003)
• Attention for learning

39. CGp: Evaluation?
• Luke: “Not determined to give up on studying regions that
have no clear role in calculating value, he decides to focus
on the posterior cingulate gyrus. However, rather than
actually describing how it might be involved in calculating
value, he gives us a list of random citations justifying how it
might play a role (sounding strangely like a grant
proposal). After reading a dense 20 page chapter on value,
I was hoping to have at least learned something about how
value might be calculated in the brain, but instead I felt like
I was reading a 20 page justification of why we should pick
random areas in the brain that are not well studied and
then not be surprised when nothing conclusive emerges.

40. CGp: Evaluation?
• Luke: “Why are goal values (i.e. motivations)
always neglected in models of value? If things
like appetitive desires (presumably processed in
the hypothalamus) are crucial for calculating goal
values, why in the world would we think the
furthest region away from them (both spatially
and evolutionarily) would be an interesting place
to begin? How might internal motivational states
(e.g. hunger, sexual drive, temperature
regulation) be integrated with external demands
(e.g. reputation)?”

41. Question
• Assuming neuronal correlates of value exist in
the OFC, LIP, and CGp, why are there so many
representations of value in the brain?
• MP & C P-S: value signals expressed in
different areas contribute to different
processes
– Frontal areas: economic choice
– Sensory areas: perceptual choice
– Motor areas: motor choice

42. Question
• Mirre: “Different neural correlates of value
have been found in the brain. In sensory
areas, value signals contribute to perceptual
attention, in frontal areas, value signals
contribute to economic choice and in motor
areas value signals contribute to action
selection. How are these signals integrated?
Which value signal determines the eventual
choice? What happens when there is a conflict
between value signals?”

43. Goods- versus Actions-based
• GBM: economic choice is modular and before
action selection
• ABM: economic choice “unfolds as a process
of action selection”
• For GBM, choice should be “completely
processed within an abstract representation of
goods” (e.g., in the OFC)

44. Other questions
• Trevor: “…Both chapters this week (and last
week) sort of handwaved at the notion that
understanding the valuation process would be
beneficial for clinical research, but never really
justified this claim. How do you think
understanding how valuation is accomplished
in the brain can inform assessment and
treatment of various clinical disorders?”

45. Other questions
• Luke “Things that are more interesting to discuss then the scientific
career of Michael Platt might include:
Is there a unitary valuation system? I think this will prove to be a hotly
contested issue. Are abstract representations somehow funneled into
the OFC or ventral striatum and then converted into a unitary
currency? Or are there independent systems involved in risk (insula,
Nacc) reward (NAcc, amygdala, OFC), and punishment (insula,
amygdala)? What about more abstract representations like future
consequences? What about motivational values originating either
exogenously or endogenously?
Is there any reason to believe that the assumptions about choice held
by economists have any validity in the brain? Is there actually a
system involved in representing value and another one for selecting the
action with the highest expected value?