Presentation at IUI2013, Santa Monica. The full paper can be found at http://bit.ly/iui2013full

1. Helping Users with Information Disclosure Decisions
Potential for Adaptation
Bart P. Knijnenburg Alfred Kobsa
Department of Informatics, UC Irvine Department of Informatics, UC Irvine

2. Mobile apps need personal data
Mobile applications often
use personalization
This requires personal
information
- Demographical data (e.g.
age, hobbies, income)
- Contextual data (e.g. app
usage, calendar, location)
INFORMATION AND COMPUTER SCIENCES

3. Justifying disclosure
Privacy problems!
Give people control
Privacy calculus:
Tradeoff between privacy
and expected benefits
Benefits are hard to grasp
Provide justifications
INFORMATION AND COMPUTER SCIENCES

4. Justification types
Explain the reason why the information is requested
May prove the legitimacy of the disclosure request
Highlight the benefits of disclosure
Privacy calculus: tip the scales in favor of the benefits
Appeal to the social norm
Eschew privacy calculus by conforming to the majority
Previous work: Justifications seem to work
Our goal: Find out which one works best
INFORMATION AND COMPUTER SCIENCES

5. Outline (spoiler alert)
Which justification works best?
Actually, “no justification” works best!
Can we do better?
We can adapt the justification (and request order) to the
user!
What does that give us?
A set of implementable heuristics!
How can we extend these findings?
Create a fully adaptive system!
INFORMATION AND COMPUTER SCIENCES

6. Experiment
INFORMATION AND COMPUTER SCIENCES

7. Experiment
INFORMATION AND COMPUTER SCIENCES

8. Manipulations
Location, etc. Gender, etc.
Gender, etc. Location, etc.
Context data first Demographical data first
INFORMATION AND COMPUTER SCIENCES

9. Manipulations
5 justification types
INFORMATION AND COMPUTER SCIENCES

10. Manipulations
5 justification types
None
INFORMATION AND COMPUTER SCIENCES

11. Manipulations
5 justification types
None
Useful for you
INFORMATION AND COMPUTER SCIENCES

12. Manipulations
5 justification types
None
Useful for you
Number of others
INFORMATION AND COMPUTER SCIENCES

13. Manipulations
5 justification types
None
Useful for you
Number of others
Useful for others
INFORMATION AND COMPUTER SCIENCES

14. Manipulations
5 justification types
None
Useful for you
Number of others
Useful for others
Explanation
INFORMATION AND COMPUTER SCIENCES

15. Which one is best?
Which increases disclosure the most?
Which increases satisfaction the most?
INFORMATION AND COMPUTER SCIENCES

16. Results
Disclosure*behavior*
*
Demographics*disclosure * *Context*disclosure*
Context#first# Demographics#first# Context#first# Demograpics#first#
100%#
90%#
80%#
70%#
60%#
50%#
40%#
30%#
20%#
10%#
0%#
INFORMATION AND COMPUTER SCIENCES

17. Results
Disclosure*behavior*
*
Demographics*disclosure * *Context*disclosure*
Context"first" Demographics"first" Context"first" Demograpics"first"
100%"
90%" 1"
80%" ***"
70%" *" **" *"
60%" *" *"
50%"
40%"
30%"
20%"
10%"
0%"
none" useful"for"you" #"of"others" useful"for"others" explanaDon"
INFORMATION AND COMPUTER SCIENCES

18. Results
Justifications are perceived Perceived(value(of(
to be helpful Disclosure*behavior* disclosure(help(
(except “# of others”)
*
Demographics*disclosure * ***"
*Context*disclosure*
Context"first" Demographics"first" 1,00" Context"first" ***"
Demograpics"first"
100%"
0,75" **"
90%" 1"
80%" ***" 0,50"
70%" 0,25" *" **" *"
60%" 0,00" *" *"
50%"
#0,25"
40%"
30%" #0,50"
20%" #0,75"
10%"
#1,00"
0%"
none" useful"for"you" #"of"others" useful"for"others" explanaDon"
INFORMATION AND COMPUTER SCIENCES

19. Results
Justifications are perceived Perceived(privacy(
to be helpful Disclosure*behavior* threat(
(except “# of others”)
*
Demographics*disclosure * *Context*disclosure*
Context"first" Demographics"first" 1,00" Context"first" Demograpics"first"
100%"
Higher privacy threat seen
90%" 1"
0,75"
*"
for “useful for others”
80%" ***" 0,50"
70%" 0,25" *" **" *"
60%" 0,00" *" *"
50%"
#0,25"
40%"
30%" #0,50"
20%" #0,75"
10%"
#1,00"
0%"
none" useful"for"you" #"of"others" useful"for"others" explanaDon"
INFORMATION AND COMPUTER SCIENCES

20. Results
Justifications are perceived Trust&in&the&&
to be helpful Disclosure*behavior* company&
(except “# of others”)
*
Demographics*disclosure * *Context*disclosure*
Context"first" Demographics"first" 1,00" Context"first" Demograpics"first"
100%"
Higher privacy threat seen
90%" 1"
0,75"
for “useful for others”
80%" ***" 0,50"
70%" 0,25" *" **" *"
Lower trust in the company
60%"
50%"
0,00" *" *"
$0,25"
40%" (esp. “useful for others”) $0,50"
30%" 1"
20%" $0,75" **"
10%"
$1,00"
0%"
none" useful"for"you" #"of"others" useful"for"others" explanaDon"
INFORMATION AND COMPUTER SCIENCES

21. Results
Justifications are perceived Sa#sfac#on)with))
to be helpful Disclosure*behavior* the)system)
(except “# of others”)
*
Demographics*disclosure * *Context*disclosure*
Context"first" Demographics"first" 1,00" Context"first" Demograpics"first"
100%"
Higher privacy threat seen
90%" 1"
0,75"
for “useful for others”
80%" ***" 0,50"
70%" 0,25" *" **" *"
Lower trust in the company
60%"
50%"
0,00" *" *"
$0,25"
40%" (esp. “useful for others”) $0,50"
30%" 1"
**" **"
Lower satisfaction
20%" $0,75"
10%"
$1,00" ***"
0%"
none" useful"for"you" #"of"others" useful"for"others" explanaDon"
INFORMATION AND COMPUTER SCIENCES

22. Problem!
Justifications did not have the expected effects
People liked the help, but:
No increase in disclosure (bad for personalization)
No decrease in perceived threat, no increase in trust
Satisfaction is lower
INFORMATION AND COMPUTER SCIENCES

23. Idea: Adaptation
I care about
the benefits
I do whatever
others do
INFORMATION AND COMPUTER SCIENCES

24. Idea: Adaptation
I am more
hesitant about
Location, etc. demographics Gender, etc.
Gender, etc. Location, etc.
I am more
hesitant about
context data
INFORMATION AND COMPUTER SCIENCES

25. Adapt to what?
Big differences in privacy behavior:
2 groups, 0-22 items and 23-31 items
3 privacy groups, based on disclosure tendency
Privacy fundamentalists, pragmatists, and unconcerned
Gender
Females have higher concerns, especially for context data
Age didn’t work
Older people have higher concerns, but are also more
careful
INFORMATION AND COMPUTER SCIENCES

26. Heuristics
Males with low disclosure tendency (73 participants)!
87%! Demograhics data! Context data!
80%! 90%! 98%!
p = .057!
Disclosure probability!
100%!
90%! 67%! p = .003!
67%! ■ Demographics first
80%! p = .003! ● Context first
70%!
60%! 45%! p = .079! 46%!
69%! 42%!
50%! p < .001! 35%!
40%! 53%! 57%!
48%! p < .001!
30%! 46%! p < .001! P = .002!
p < .001! P = .004!
20%! p < .001! 30%! 29%! 30%! ! !
10%! p = .055! p = .076! p = .054! 17%! 14%!
0%!
no! useful! number ! useful for! expla-! no! useful! number ! useful for! expla-!
justification! for you! of others! others! nation! justification! for you! of others! others! nation!
Demographics disclosure
Females with low disclosure tendency (88 data disclosure
Context participants)!
- Demographics first, - Context first, no
Demograhics data! Context data!
Disclosure probability!
100%!
90%! 81%! 79%! 75%!
57%!
‘explanation’ justification justification
80%! p = .027! 89%!
70%!
51%! 47%! 51%!
60%! 46%!
50%! 66%! 64%! 40%!
60%! 58%!
40%! ! p < .001!
Overall disclosure: Context first, ‘useful for < .001! justification
p you’
p < .001! p < .001! 48%! p = .010! p = .009!
30%! p = .005! p < .001! !
20%! ! p < .001! ! 36%!
!
27%!
10%! 23%! 20%!
0%! 11%!
no! useful! number ! useful for! expla-! no! useful! number ! useful for! expla-!
INFORMATION AND COMPUTER SCIENCES
justification! for you! of others! others! nation! justification! for you! of others! others! nation!

27. Heuristics
Satisfaction
- Context first, ‘explanation’ justification
- Demographics first ‘useful for you’ justification
Males with low disclosure tendency (73 participants)
■ Demographics first
● Context first
Perceived disclosure help! Perceived privacy threat! Trust in the company! Satisfaction!
2,0! 1,5! p=.063 1,5! 1,0!
p=.023
1,5! 1,0! 1,0! 0,5!
1,0! 0,5! p=.089 0,5! 0,0! p=.097
p=.052 p=.066
0,5! 0,0! 0,0! -0,5!
p=.045 p=023
0,0! -0,5! -0,5! -1,0! p=.009 p=.008 p=.012
p=.006
-0,5! =.009 -1,0! -1,0! p=.036
p=.038 -1,5!
-1,0! -1,5! -1,5! -2,0!
useful! usef. useful! usef. useful! usef. useful! usef.
no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.!
just.! you! othrs! othrs! just.! you! othrs! othrs! just.! you! othrs! othrs! just.! you! othrs! othrs!
INFORMATION AND COMPUTER SCIENCES

28. Heuristics -- best compromise
Males with low disclosure tendency (73 participants)!
87%! Demograhics data! Context data!
80%! 90%! 98%!
p = .057!
Disclosure probability!
100%!
90%! 67%! p = .003!
67%! ■ Demographics first
80%! p = .003! ● Context first
70%!
60%! 45%! p = .079! 46%!
69%! 42%!
50%! p < .001! 35%!
40%! 53%! 57%!
48%! p < .001!
30%! 46%! p < .001! P = .002!
p < .001! P = .004!
20%! p < .001! 30%! 29%! 30%! ! !
10%! p = .055! p = .076! p = .054! 17%! 14%!
0%!
no! useful! number ! useful for! expla-! no! useful! number ! useful for! expla-!
justification! for you! of others! others! nation! justification! for you! of others! others! nation!
Males with low disclosure tendency (73 participants)
Perceived disclosure help! Females with low disclosure tendency (88 participants)!
Perceived privacy threat! Trust in the company! Satisfaction!
2,0! 1,5! 1,5! 1,0!
Demograhics data! p=.063 Context data!
p=.023
Disclosure probability!
100%!
1,5! 1,0! 1,0! 0,5!
90%! 81%! 79%! 75%!
57%! p=.089
1,0! 80%! 0,5! 0,5! 0,0! p=.097
p = .027! 89%!
p=.052 p=.066
0,5! 70%! 0,0! 0,0! 51%! -0,5! 51%!
60%!
p=.045 46%! 47%!
p=023 40%!
0,0! 50%! 66%! -0,5! 64%! -0,5! -1,0! p=.009 p=.008 p=.012
60%! 58%!
40%! ! p=.006 p < .001! p = .010! p = .009!
-0,5! 30%! p = .005! p < .001! -1,0!< .001!
p 48%! -1,0! -1,5! p=.036
=.009 p=.038 p < .001! !
20%! p < .001! ! p < .001! ! 36%!
-1,0! -1,5! -1,5! ! -2,0! 27%!
10%! useful! usef. useful! usef. 23%!
useful! usef. 20%! useful! usef.
no! for # of! for expl.!
0%! no! for # of! for expl.! no! for # of! 11%! expl.!
for no! for # of! for expl.!
just.! you!no! useful!
othrs! othrs! just.! !
number you!useful for!
othrs! othrs! expla-! just.!no! useful!
you! othrs! othrs! number just.!useful for! othrs!
! you! othrs! expla-!
INFORMATION AND COMPUTER SCIENCES
justification! for you! of others! others! nation! justification! for you! of others! others! nation!

29. 50%! 35%!
Disclosure
p < .001! 57%!
40%! 53%!
48%! p < .001!
30%! 46%! p < .001! P = .002!
p < .001! P = .004!
20%! p < .001! 30%! 29%! 30%! !
Heuristics
!
10%! p = .055! p = .076! p = .054! 17%! 14%!
0%!
no! useful! number ! useful for! expla-! no! useful! number ! useful for! expla-!
justification! for you! of others! others! nation! justification! for you! of others! others! nation!
Males with low disclosure tendency (73 participants)
Perceived disclosure help! Females with low disclosure tendency (88 participants)!
Perceived privacy threat! Trust in the company! Satisfaction!
2,0! 1,5!
Demograhics data! p=.063 1,5! 1,0!
Context data!
p=.023
Disclosure probability!
100%!
1,5! 1,0! 1,0! 0,5!
90%! 57%!
81%! 79%! 75%! ■ Demographics first
1,0! 80%!
p = .027!
0,5!
89%!
p=.089 0,5! 0,0! ● Context first
p=.097
70%! p=.052 p=.066
0,5! 0,0! 0,0! 51%! -0,5! 51%!
60%!
p=.045 46%! 47%!
p=023 40%!
0,0! 50%! 66%! -0,5! 64%! -0,5! -1,0! p=.009 p=.008 p=.012
60%! 58%!
40%! ! p=.006 p < .001! p = .010! p = .009!
-0,5! 30%! p = .005!
p < .001! -1,0!< .001!
p 48%! -1,0! -1,5! p=.036
=.009 p=.038 p < .001! !
20%! p < .001! ! p < .001! ! 36%!
-1,0! -1,5! -1,5! ! -2,0! 27%!
10%! useful! usef. useful! usef. 23%!
useful! usef. 20%! useful! usef.
0%! for # of! for expl.!
no! no! for # of! for expl.! no! for # of! for 11%! expl.! no! for # of! for expl.!
just.! you!no! useful!
othrs! othrs! number you!useful for!
just.! ! othrs! othrs! expla-! just.!no! useful!
you! othrs! othrs! number just.!useful for! othrs!
! you! othrs! expla-!
justification! for you! of others! others! nation! justification! for you! of others! others! nation!
Females with low disclosure tendency (88 participants)
Perceived disclosure help! Perceivedwith high disclosure tendency (150 participants)!
Males privacy threat! Trust in the company! Satisfaction!
2,0! 1,5!
Demograhics data! 1,5! 1,0!
Context data!
99%! 98%! 97%! 100%! 99%!
Disclosure probability!
1,5! 100%! 1,0! p=.012 1,0! 0,5! 87%! 76%!
78%!
!
90%! p=.036
p = .063! p=.036 78%! 88%! p = .041!
1,0! 97%! 0,5!95%! 98%! p = .055!
80%! 95%!
p=.006 p = .055! 93%! 0,5! 0,0!
70%! p = .012! p = .014! p = .099! 85%!
0,5! 0,0! p = .002!
0,0!
82%! 80%!
-0,5!
60%! p=.026 70%! p=.061 72%!
0,0! 50%! -0,5! -0,5! -1,0! p = .014!
p = .004! p=.045
40%! p<.001
-0,5! 30%! -1,0! -1,0! -1,5!
p<.001
20%!
-1,0! -1,5! -1,5! -2,0!
10%! useful! usef. useful! usef. useful! usef. useful! usef.
0%! for # of! for expl.!
no! no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.!
no! othrs!useful!
just.! you! othrs! number you! useful for!
just.! ! othrs! othrs! expla-! no! useful!
just.! you! othrs! othrs! number !just.! you!for!
useful othrs! othrs!
expla-!
justification! for you! of others! others! nation! justification! for you! INFORMATION ANDothers! SCIENCES
of others! COMPUTER nation!

30. 0,0! 50%! 66%! -0,5! 64%! -0,5! 40%!
-1,0! p=.009 p=.008 p=.012
Disclosure
60%! 58%!
40%! ! p=.006 p < .001! p = .010! p = .009!
-0,5! 30%! p = .005! p < .001! -1,0!< .001!
p 48%! -1,0! -1,5! p=.036
=.009 p=.038 p < .001! !
20%! p < .001! ! p < .001! ! 36%!
27%!
Heuristics
-1,0! -1,5! -1,5! ! -2,0!
10%! useful! usef. useful! usef. 23%!
useful! usef. 20%! useful! usef.
0%! for # of! for expl.!
no! no! for # of! for expl.! no! for # of! 11%! expl.!
for no! for # of! for expl.!
just.! you!no! useful!
othrs! othrs! just.! !
number you!useful for!
othrs! othrs! expla-! just.!no! useful!
you! othrs! othrs! number just.!useful for! othrs!
! you! othrs! expla-!
justification! for you! of others! others! nation! justification! for you! of others! others! nation!
Females with low disclosure tendency (88 participants)
Perceived disclosure help! Perceivedwith high disclosure tendency (150 participants)!
Males privacy threat! Trust in the company! Satisfaction!
2,0! 1,5!
Demograhics data! 1,5! 1,0!
Context data!
99%! 98%! 97%! 100%! 99%!
p=.012
Disclosure probability!
1,5! 100%! 1,0! ! 1,0! 78%! 0,5! 87%! 76%!
90%! p=.036
p = .063! p=.036 78%! 88%! p = .041!
1,0! 97%! 0,5!95%! 98%! p = .055!
80%! 95%! p=.006 p = .055! 93%! 0,5! 0,0!
70%! p = .012! p = .014! p = .099! 85%!
0,5! 0,0! p = .002!
0,0! 82%! 80%!
-0,5!
60%! p=.026 70%! p=.061 72%!
0,0! 50%! -0,5! -0,5! -1,0! p = .014!
p = .004! p=.045
40%! p<.001
-0,5! 30%!
p<.001
-1,0! -1,0! -1,5!
■ Demographics first
-1,0!
20%!
10%! useful! usef.
-1,5!
useful! usef.
-1,5!
useful! usef.
-2,0! ● useful! first
Context usef.
0%! for # of! for expl.!
no! no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.!
just.! you! no! othrs!useful!
othrs! number you! useful for!
just.! ! othrs! othrs! expla-! no! useful!
just.! you! othrs! othrs! number !just.! you!for!
useful othrs! othrs!
expla-!
justification! for you! of others! others! nation! justification! for you! of others! others! nation!
Males with high disclosure tendency (150 participants)
Perceived disclosure help! Females with high disclosure tendency (178 participants)!
Perceived privacy threat! Trust in the company! Satisfaction!
2,0! 1,5!
Demograhics data! 99%! 1,5!
95% ! 1,0!
Context data!
99%! 99%! 96% ! p = .024!
Disclosure probability!
1,5! 100%! 1,0! 1,0! 88%! 86%! 0,5! 83%! 84%!
p=.056
!
p = .049! 71% !
90%!
1,0! 80%! p=.011
95%! 0,5! 96%! 96%! 0,5! p = .003!
0,0!
p=.001 p=.032
91%! p=.003
92%!
p=.049 p = .069!
0,5!
70%! p = .029! p = .002! p = .001! p=.039= .046!
0,0!
p p=.032
0,0! -0,5!
p=.047
60%! p=.061 71%! 75%! 75%!
p<.001 p<.001 68%!
0,0! 50%! -0,5! p=.060 -0,5! p = .004! p = .016! 65%!
-1,0! p = .018!
p < .001! p = .001!
40%!
-0,5! 30%! -1,0! -1,0! -1,5!
20%!
-1,0! -1,5! -1,5! -2,0!
10%! useful! usef. useful! usef. useful! usef. useful! usef.
0%! for # of! for expl.!
no! no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.!
no! othrs!useful!
just.! you! othrs! number you! useful for!
just.! ! othrs! othrs! expla-! no! useful!
just.! you! othrs! othrs! number !just.! you!for!
useful othrs! othrs!
expla-!
justification! for you! of others! others! nation! justification! for you! INFORMATION ANDothers! SCIENCES
of others! COMPUTER nation!

31. 0,0! 50%! -0,5! -0,5! -1,0! p = .004! p=.045
p = .014!
Disclosure
40%! p<.001
-0,5! 30%! -1,0! -1,0! -1,5!
p<.001
20%!
Heuristics
-1,0! -1,5! -1,5! -2,0!
10%! useful! usef. useful! usef. useful! usef. useful! usef.
0%! for # of! for expl.!
no! no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.!
just.! you! no! othrs!useful!
othrs! just.! !
number you! useful for!
othrs! othrs! expla-! just.! you! othrs! othrs!
no! useful! number !just.! you!for!
useful othrs! expla-!
othrs!
justification! for you! of others! others! nation! justification! for you! of others! others! nation!
Males with high disclosure tendency (150 participants)
Perceived disclosure help! Females with high disclosure tendency (178 participants)!
Perceived privacy threat! Trust in the company! Satisfaction!
2,0! 1,5!
Demograhics data! 99%! 1,5!
95% ! 1,0!
Context data!
99%! 99%! 96% ! p = .024!
Disclosure probability!
1,5! 100%! p=.056 1,0! ! 1,0! 88%! 86%! 0,5!
83%! 84%!
p = .049! 71% !
90%!
1,0! p=.011 0,5! p=.003 96%! 96%! 0,5! p = .003!
0,0!
80%! 95%!
p=.001 p=.032
91%! 92%!
p=.039= .046!
p p=.049 p = .069!
p=.032
0,5!
70%! p = .029! p = .002! 0,0! = .001!
p 0,0! -0,5!
p=.047
60%! p<.001 p=.061 71%! 75%! 75%!
p<.001 p = .016! 68%! p = .018!
0,0! 50%! -0,5! p=.060 -0,5! p = .004! 65%!
-1,0!
p < .001! p = .001!
40%!
-0,5! 30%! -1,0! -1,0! -1,5! ■ Demographics first
-1,0!
20%!
-1,5! -1,5! -2,0! ● Context first
10%! useful! usef. useful! usef. useful! usef. useful! usef.
0%! for # of! for expl.!
no! no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.!
just.! you! no! othrs!useful!
othrs! number you! useful for!
just.! ! othrs! othrs! expla-! no! useful!
just.! you! othrs! othrs! number !just.! you!for!
useful othrs! othrs!
expla-!
justification! for you! of others! others! nation! justification! for you! of others! others! nation!
Females with high disclosure tendency (178 participants)
Figure 3: disclosure help! justification type and request order on disclosure (for each type of data, gender, and disclosure tendency)
Perceived
The effects of Perceived privacy threat! Trust in the company! Satisfaction!
Error bars are ±1 standard error.1,5! best strategy is labeled with an arrow; strategies with a p-value perform significantly worse
2,0! The 1,5! 1,0!
1,5! 1,0! 1,0! 0,5!
p=.001
1,0! p=.032 0,5! p=.002 0,5! 0,0!
p=.058 p=.025
p=.074 p=.039
0,5! 0,0! 0,0! -0,5!
p=.001 p=.049 p=.059
p=.003 p=.039 p=.041
0,0! -0,5! -0,5! -1,0!
-0,5! -1,0! -1,0! -1,5!
-1,0! -1,5! -1,5! -2,0!
useful! usef. useful! usef. useful! usef. useful! usef.
no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.! no! for # of! for expl.!
just.! you! othrs! othrs! just.! you! othrs! othrs! just.! you! othrs! othrs! just.! you! othrs! othrs!
INFORMATION AND COMPUTER SCIENCES

32. Following the heuristics
How to discover gender?
- Can just be the first item to request
- Highest disclosure rate of all items in our study (94.9%)
How to discover disclosure tendency?
- Make some (potentially invasive) requests (not desirable)
- Ask about (or otherwise determine) stated privacy
concerns, mobile Internet usage and/or tech-savvyness
INFORMATION AND COMPUTER SCIENCES

33. Future work
Test the heuristics in a real system
And not just an app recommender
Find the rationale behind the heuristics
E.g. why do males and females require different
justifications?
Make the system even more adaptive
- Select the optimal justification for each item
- Order the requests dynamically to ensure the highest
possible disclosure (without reducing satisfaction)
INFORMATION AND COMPUTER SCIENCES

34. Conclusion (you already know this)
Which justification works best?
Actually, “no justification” works best!
Can we do better?
We can adapt the justification (and request order) to the
user!
What does that give us?
A set of implementable heuristics!
How can we extend these findings?
Create a fully adaptive system!
INFORMATION AND COMPUTER SCIENCES

35. Thank you
bart.k@uci.edu :: www.usabart.nl :: @usabart