My Keynote from the Social Recommendation Systems workshop for CSCW2011.

1. Recommending Actions,
Not Content
david @ayman shamma
internet experiences
microeconomics & social systems

2. Internet Experiences Group
(David) Ayman
Shamma Lyndon Kennedy Jude Yew Elizabeth Churchill

3. Disclaimer: I !<3 Recommendation Systems

4. Disclaimer: I !<3 Recommendation Systems
I <3 Engagement

8. #FAIL

14. #FAIL

16. Really? What are we doing?

17. Really? What are we doing?
What are we recommending?
Why are we doing that?

20. Image Search

21. Image Search

22. Image Search A Text
Box!!!!

23. Search as Recommendation

25. Play the
Music

26. Click Through on Search Pages
BoIom of
“fold”
BoIom of
Page
BoIom of
2nd Window
Adapted from “A Dynamic Bayesian Network Click Model for Web Search Ranking,” by Olivier Chapelle, Ya Zhang, WWW’09.

27. Is Recent a Relevant Recommendation?

28. Recent
Normal

30. Does Relevance Matter?
• Bottom of the page
– Normally low click through
– Show alternate results

31. Does Relevance Matter?
• Bottom of the page
– Normally low click through
– Show alternate results

32. Does Relevance Matter?
• Bottom of the page
– Normally low click through
G
WRON
– Show alternate results

33. Does Relevance Matter?
• Bottom of the page Precision/recall
– Normally low click through
G doesn’t (always)
WRON matter!!
– Show alternate results
(for multimedia)

34. Un-related images at the bottom of the page
should be here.
BoIom of
“fold”
BoIom of
Page
BoIom of
2nd Window
Adapted from “A Dynamic Bayesian Network Click Model for Web Search Ranking,” by Olivier Chapelle, Ya Zhang, WWW’09.

35. Un-related images at the bottom of the page
are here!!!
BoIom of
“fold”
BoIom of
Page
BoIom of
2nd Window
Adapted from “A Dynamic Bayesian Network Click Model for Web Search Ranking,” by Olivier Chapelle, Ya Zhang, WWW’09.

36. What’s Similar? Have a listen.
Song 1 Song 2
Song 3
32

37. Song 1

38. Song 1

39. Song 2

40. Song 2

41. Song 3

42. Song 3

43. Context
Song
Rater

44. Context
Song
Rater

45. Context
Song
Rater

46. So what do you like?
Song 1 Song 2
Song 3
37

47. Song 1

48. Song 1

49. Song 2

50. Song 2

51. Song 3

52. Song 3

53. Think about ratings

54. Song Similarity Example
Song 1 Song 2 Song 3
Jazz Lover 5 0 5
Rock Lover 5 0 5
Classical Lover 0 5 0

55. Song Similarity Example
Song 1 Song 2 Song 3
Jazz Lover 5 0 5
Rock Lover 5 0 5
Classical Lover 0 5 0
Similar Songs

56. A Small Experiment (by M. Slaney)
• 380,911 Subjects
• 1000 Jazz Songs
• 1,449,335 Rabngs
Never Play this Again Love It!

57. Users do not rate everything….
Self‐Selected Rabng Histogram True Rabng Histogram
(1.5B rabngs) (350k rabngs)
From: Marlin, Zemel, Roweis, Slaney. “Collaborabve Filtering and the missing at random assumpbon.” UAI 2007

58. About the Data
46

59. About the Data
• Real rabng data
– Y! Music
Y! Data
– 700M rabngs
46

60. About the Data
• Real rabng data
– Y! Music
Y! Data
– 700M rabngs
True Distribubon
46

61. About the Data
• Real rabng data
– Y! Music
Y! Data
– 700M rabngs
d of
o
True Distribubon l iho
ike ing
L y
pla
46

62. Neilix Compebbon
• Create new recommendabon algorithm
– 10% beIer than Neilix algorithm
• Data
– 100M rabngs
– 480k users, 17k movies
• Winner
– BellKorPragmabcChaos

63. Movie rabng data
Training data Test data
• Training data score movie user movie user
– 100 million rabngs 1 21 1 ? 62 1
5 213 1 ? 96 1
– 480,000 users
4 345 2 ? 7 2
– 17,770 movies
4 123 2 ? 3 2
– 6 years of data:
3 768 2 ? 47 3
2000‐2005
5 76 3 ? 15 3
• Test data
4 45 4 ? 41 4
– Last few rabngs of
1 568 5 ? 28 4
each user (2.8
2 342 5 ? 93 5
million)
2 234 5 ? 74 5
• Dates of rabngs are
5 76 6 ? 69 6
given
4 56 6 ? 83 6

64. Components of a rabng predictor
user bias movie bias user‐movie interacbon
Baseline predictor User‐movie interacbon
• Separates users and movies • Characterizes the matching
• Onen overlooked between users and movies
• Benefits from insights into users’ • AIracts most research in the field
behavior
• Benefits from algorithmic and
• Among the main pracbcal
contribubons of the compebbon mathemabcal innovabons
Courtesy of YehudaKoren

65. This is kinda why we are here...

66. Legacy Video

67. Traditional Comments and Tags
Left in Whole, Unattached.

68. Quickly...let me tell you why I hate tags...

69. Tag this.

70. Tag this.

71. Tag This

72. Tag Noise

73. Who’s Christmas?
Canada Australia

74. Hey aren’t categories tags anyhow?

75. Double Rainbow Pick a category

76. Anyway, back on track...

77. Social Conversations Happen Around Media
Dolores Park, San Francisco, 2006

78. Social Conversations Happen Around Media
Dolores Park, San Francisco, 2006

79. Social Conversations happen around videos
Well – actually people join in a session and converse afterwards.

81. What to Collect to measure
• Type of event
(Zync player command or a normal chat message)
• Anonymous hash
(uniquely identifies the sender and the receiver, without
exposing personal account data)
• URL to the shared video
• Timestamp for the event
• The player time (with respect to the specific video) at the
point the event occurred
• The number of characters and the number words typed
(for chat messages)
• Emoticons used in the chat message

82. A Short Movie

83. Percent of actions over time.

84. Chat follows the video!
CHAT

85. http://www.flickr.com/photos/wvs/3833148925/

86. Reciprocity
• 43.6% of the sessions the invitee played at
least one video back to the session’s initiator.
• 77.7% sharing reciprocation
• Pairs of people often exchanged more than
one set of videos in a session.
• In the categories of Nonprofit, Technology
and Shows, the invitees shared more videos

87. How do we know what people are watching?
How can we give them better things to watch?
CLASSIFICATION

88. Types of features on YouTube

89. 5 star ratings has been the golden egg for recommendation systems
so far; implicit human cooperative sharing activity works better.
CLASSIFICATION BASED ON
IMPLICIT CONNECTED SOCIAL

90. 20 random videos sent to 43 people.
60.3% identified the category correctly.
52.3% identified the comedies correctly.
PEOPLE REALLY STINK AT THIS

91. Used and Unused Data
You Tube Zync
Duration (video) Duration (session)*
Views (video)
Duration # of Play/Pause*
Duration (session)*
Rating*
Views # of Scrubs*
# of Play/Pause*
Rating* # of Chats*
# of Scrubs*
You Tube (not used) Zync (not used)
Tags Emoticons
Comments User ID data
Favorites # of Sessions
# of Loads

92. Phone in your favorite ML technique.
FIRST ORDER DATA WASN’T
PRETTY

93. Naïve Bayes Classification
Type Accuracy
Random Chance 23.0%
You Tube Features 14.6%
You Tube Top 5 Categories 32.4%
Zync Features 53.9%
Humans 60.9%

94. What about these three videos? Which one you like?
Nominal Factorization

95. Ratings doen’t particularly specify order.
Nominal Factorization

96. Classification with Factoring
Type Accuracy
Random Chance 23.0%
You Tube Features 14.6%
You Tube Top 5 Categories 32.4%
YT Top 5 Factoring Duration 51.8%
Humans 60.9%
YT Top 5 Factoring Views 66.9%
YT Top 5 Factoring Ratings 75.5%
YT Top 5 Factoring All Features 75.9%
psst, yes we know that more training will do the same thing eventually,
I just don’t like waiting.

97. Classification w/ Zync features
Type Accuracy
Random Chance 23.0%
You Tube Features 14.6%
You Tube Top 5 Categories 32.4%
YT Top 5 Factoring Duration 51.8%
Humans 60.9%
YT Top 5 Factoring Views 66.9%
YT Top 5 Factoring Ratings 75.5%
YT Top 5 Factoring All Features 75.9%
Zync Factored All Features 87.8%
psst, we are looking at using Gradient Boosted Decision Trees in our
future work.

98. Finding the viral.
Can we predict if a video has over 10M views?
More so, can we do so with say 10 people across 5 sessions?

99. Remember this is what
we have for data

100. Viral Classification w/ Zync features
Does the video have over 10 M views? Accuracy
Guessing Yes 6.3%
Guessing No 93.7%
Guessing Randomly 88.3%
Naive Bayes (25% training set) 89.2%
Naive Bayes (50% training set) 95.5%
Naive Bayes (80% training set) 96.6%

101. Three pieces
Classifier
Survey Data Interviews

102. Audience Perception
Just ask Homer
is Key

103. I !<3 Recommendation Systems

104. 3 areas prime for social recommendation for disrupt:

105. 1: Understanding the temporal and the recent.

106. Social Conversations Happen Around Media
Dolores Park, San Francisco, 2006

107. Social Conversations Happen Around Media
Dolores Park, San Francisco, 2006

108. Come see my talk!

109. Lets find a moment Here’s an example.

110. All Tweets Inauguration Tweets
Left: All tweet sample.
Right: Tweets with Inauguration keywords.

111. All Tweets Inauguration Tweets All Tweets with @
Left: All tweet sample.
Right: Tweets with Inauguration keywords.

112. 12:04 is what you want to
watch.

113. 2: Q & A

115. Likes
Generalization
Questioning
the Question
Clarification
One Answer
Finding answers... ...kinda like Watson.

116. 3: Challenges

117. Me: You’re in China, go to the night market for !!

118. Me: You’re in China, go to the night market for !!
My friend: Street food? Are you kidding? I’ll get sick!

119. Me: You’re in China, go to the night market for !!
My friend: Street food? Are you kidding? I’ll get sick!
Me: I dare you not to!!

120. Me: You’re in China, go to the night market for !!
You: Street food? Are you kidding? I’ll get sick!
Me: I dare you not to! (It’s delicious!)

121. Man vs. Food http://www.travelchannel.com/TV_Shows/
Man_V_Food

122. Why try to understand engagement?
Better advertising.
Better understanding of the relationship between users and the sharing/
consumption of media content.
Better organization and classification of media for efficient navigation and content
retrieval.
Better recommendations!

123. Find me: @ayman • aymans@acm.org
Fin & Thanks!
Thanks to D. DuBois, M. Slaney, E. Churchill, L. Kennedy, J.Yew, S. Pentland, A.
Brooks, J. Dunning, B. Pardo, M. Cooper.
Knowing Funny: Genre Perception and Categorization in Social Video Sharing Jude Yew; David A. Shamma; Elizabeth F.
Churchill, CHI 2011, ACM, 2011
Peaks and Persistence: Modeling the Shape of Microblog Conversations David A. Shamma; Lyndon Kennedy; Elizabeth F.
Churchill, CSCW 2011, ACM, 2011
In the Limelight Over Time: Temporalities of Network Centrality David A. Shamma; Lyndon Kennedy; Elizabeth F. Churchill,
CSCW 2011, ACM, 2011
Tweet the Debates: Understanding Community Annotation of Uncollected Sources David A. Shamma; Lyndon Kennedy;
Elizabeth F. Churchill, ACM Multimedia, ACM, 2009
Understanding the Creative Conversation: Modeling to Engagement David A. Shamma; Dan Perkel; Kurt Luther, Creativity and
Cognition, ACM, 2009
Spinning Online: A Case Study of Internet Broadcasting by DJs David A. Shamma; Elizabeth Churchill; Nikhil Bobb; Matt
Fukuda, Communities & Technology, ACM, 2009
Zync with Me: Synchronized Sharing of Video through Instant Messaging David A. Shamma; Yiming Liu; Pablo Cesar, David
Geerts, Konstantinos Chorianopoulos, Social Interactive Television: Immersive Shared Experiences and Perspectives,
Information Science Reference, IGI Global, 2009
Enhancing online personal connections through the synchronized sharing of online video Shamma, D. A.; Bastéa-Forte, M.;
Joubert, N.; Liu, Y., Human Factors in Computing Systems (CHI), ACM, 2008
Supporting creative acts beyond dissemination David A. Shamma; Ryan Shaw, Creativity and Cognition, ACM, 2007
Watch what I watch: using community activity to understand content David A. Shamma; Ryan Shaw; Peter Shafton; Yiming
Liu, ACM Multimedia Workshop on Multimedia Information Retrival (MIR), ACM, 2007
Zync: the design of synchronized video sharing Yiming Liu; David A. Shamma; Peter Shafton; Jeannie Yang, Designing for
User eXperiences, ACM, 2007