Movie vs movie
•Télécharger en tant que PPT, PDF•
2 j'aime•1,758 vues
What are your top ten favorite movies of all time? This is a very difficult question. But why? Irmak Sirer explains the challenges of measuring how much we like movies, books, songs, or products; combining insights from diverse sources like the Netflix Prize, Duncan Watts' social experiments, or the beginnings of Facebook. The better we get at measuring and ranking levels of enjoyment, the better we can customize websites, sort search results, find other people with similar tastes, and recommend products, so can we overcome these challenges? Drumroll... Yes, we can.
Recommandé
Contenu connexe
Tendances
Tendances (16)
En vedette
En vedette (20)
Similaire à Movie vs movie
Similaire à Movie vs movie (20)
Dernier
Dernier (20)
Movie vs movie
- 4. AGE 7 Oh cool. Pretty good. Space and stuff.
- 6. AGE 14 Omigod Omigod Omigod. Epic masterpiece is epic!!!!1! I'm in love with Leia.
- 8. AGE 17 WTF?
- 10. AGE 30 When you think about it, it's not that good.
- 11. AGE 30 When you think about it, it's not that good. Ah, who am I kidding? It's amazing. I'm still in love with Leia.
- 12. I mean... look at her.
- 13. What determines how much I like a movie?
- 14. What determines how much I like a movie? Is my reaction to a movie / book / song predictable?
- 17. How much will I like The Book of Eli?
- 19. Cinematch I have a soulmate in taste Irmak
- 20. Cinematch I have a soulmate in taste Irmak Frrmack
- 21. Cinematch I have a soulmate in taste Watched the same movies Irmak Frrmack
- 22. Cinematch I have a soulmate in taste Watched the same movies Gave the exact same ratings Irmak Frrmack
- 23. Cinematch I have a soulmate in taste Watched the same movies Gave the exact same ratings Except The Book of Eli Irmak Frrmack
- 24. Cinematch I have a soulmate in taste Frrmack watched The Book of Eli Irmak Frrmack
- 25. Cinematch I have a soulmate in taste Oh man, it was… Irmak Frrmack
- 26. Cinematch I have a soulmate in taste Oh man, it was… FANTASTIC! Irmak Frrmack
- 27. Cinematch I have a soulmate in taste Oh man, it was… FANTASTIC! Predict Irmak Frrmack
- 28. No perfect soulmates in real life Irmak
- 29. No perfect soulmates in real life Almost soulmate 1 Irmak
- 30. No perfect soulmates in real life Almost soulmate 1 Irmak Almost soulmate 2
- 31. No perfect soulmates in real life Almost soulmate 1 Irmak Almost soulmate 3 Almost soulmate 2
- 32. No perfect soulmates in real life Almost soulmate 1 Irmak Almost soulmate 2 Almost soulmate 3 Almost soulmate 4
- 33. No perfect soulmates in real life 87% soulmate Irmak 74% soulmate 82% soulmate 95% soulmate
- 34. No perfect soulmates in real life Irmak
- 35. No perfect soulmates in real life Irmak
- 36. Cinematch Works well for movies that everybody rates
- 37. Cinematch Quite bad with movies that only few people rate
- 38. Cinematch Some movies are especially difficult to predict Biggest error source: popular but weird 15% of all errors from ONE movie
- 40. Trivial: Mean score of everyone
- 41. Trivial: Mean score of everyone Error: (RMSE) 1.0540 stars
- 42. Trivial: Mean score of everyone Error: (RMSE) 1.0540 stars Cinematch Error: (RMSE) 0.9525 stars
- 43. Trivial: Mean score of everyone Error: (RMSE) 1.0540 stars 9.6% Cinematch Error: (RMSE) 0.9525 stars
- 44. Trivial: Mean score of everyone Error: (RMSE) 1.0540 stars 9.6% Cinematch Error: (RMSE) 0.9525 stars Better rankings Better recommendations
- 45. Trivial: Mean score of everyone Error: (RMSE) 1.0540 stars 9.6% Cinematch Error: (RMSE) 0.9525 stars Better rankings Better recommendations + 8.6% + 1200% people watch top recommendation BigChaos Netflix Prize Report
- 47. Cinematch Error: 0.9525 stars $1,000,000 for a 10% improvement 2006
- 48. Cinematch Error: 0.9525 stars Bring it down to: Error: 0.8563 stars $1,000,000 for a 10% improvement 2006
- 50. How did they do it?
- 51. How did they do it?
- 52. How did they do it? Before: Solid assumptions You have a certain taste. Your taste dictates a hidden rating for Book of Eli. When you watch it, this rating is revealed to you.
- 53. How did they do it? Before: Solid assumptions G N O R W You have a certain taste. Your taste dictates a hidden rating for Book of Eli. When you watch it, this rating is revealed to you.
- 54. How did they do it? After: Your rating changes with time.
- 55. How did they do it? After: Your rating changes with time. It depends on...
- 56. How did they do it? After: Your rating changes with time. It depends on... how many you rated that day your average rating for the day which movies you rated on this day shown Netflix prediction
- 57. Trivial: Mean score of everyone Error: 1.0540 stars Cinematch Error: 0.9525 stars Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
- 58. Trivial: Mean score of everyone Error: 1.0540 stars Cinematch Error: 0.9525 stars Your time dependent rating tendencies Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
- 59. Trivial: Mean score of everyone Error: 1.0540 stars Cinematch Error: 0.9525 stars Your time dependent rating tendencies Error: 0.9278 stars Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
- 60. Trivial: Mean score of everyone Error: 1.0540 stars Cinematch Error: 0.9525 stars 12.0% Your time dependent rating tendencies Error: 0.9278 stars Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
- 61. Trivial: Mean score of everyone Error: 1.0540 stars Cinematch Error: 0.9525 stars 12.0% Your time dependent rating tendencies Error: 0.9278 stars without looking at which movies you like/hate! Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
- 62. What does this suggest?
- 63. What does this suggest? We cannot compare a movie with all others we've seen.
- 64. What does this suggest? We cannot compare a movie with all others we've seen. We compare it to a limited set.
- 65. What does this suggest? We cannot compare a movie with all others we've seen. We compare it to a limited set. Liking (real time & remembered) depends on time and mood.
- 66. What does this suggest? We cannot compare a movie with all others we've seen. We compare it to a limited set. Liking (real time & remembered) depends on time and mood. Other people's opinions affect our own (followers / hipsters)
- 67. What does this suggest? We cannot compare Book of Eli with all movies we've seen. We compare it to a limited set. Liking (real time & remembered) depends on time and mood. Other people's opinions affect our own (followers / hipsters)
- 68. An experiment Music Lab: A website for downloading music
- 69. An experiment Same website: Music download and rating M.J. Salganik, P.S. Dodds, D.J. Watts. Science, 311:854-856, 2006
- 70. An experiment Music Lab: A website for downloading music Alternative A: Other people's ratings invisible
- 71. An experiment Music Lab: A website for downloading music Alternative A: Other people's ratings invisible More or less equal ratings
- 72. An experiment Music Lab: A website for downloading music Alternative A: Other people's ratings invisible Alternative B: All ratings visible More or less equal ratings
- 73. An experiment Music Lab: A website for downloading music Alternative A: Other people's ratings invisible Alternative B: All ratings visible More or less equal ratings Several songs snowball in popularity
- 74. An experiment Music Lab: A website for downloading music Alternative A: Other people's ratings invisible Alternative B: All ratings visible More or less equal ratings Several songs snowball in popularity It's different songs for each trial
- 75. Social influence plays a big part in determining hits and misses
- 76. Problems with rating movies We cannot compare a movie with all others we've seen. We compare it to a limited set. Liking (real time & remembered) depends on time and mood. Other people's opinions affect our own.
- 77. Degree of liking is sensitive and vague Amazing! Tuesday 3am Total garbage Sunday 12pm
- 78. Degree of liking is sensitive and vague Liking (real time & remembered) depends on time and mood. Other people's opinions affect our own.
- 79. Degree of liking is sensitive and vague Dependent on many other environmental factors besides our taste
- 80. Degree of liking is sensitive and vague We cannot compare a movie with all others we've seen. We compare it to a limited set.
- 81. Degree of liking is sensitive and vague Difficult to describe accurately and consistently with a number
- 82. Predicting aside, can I even reliably rate & rank movies I’ve seen in terms of enjoyment?
- 83. What are your top twenty movies? Irmak Frrmack
- 84. What are your top twenty movies? Well… Ummm… Irmak Frrmack
- 85. What are your top twenty movies? Well… Ummm… I like Star Wars. Irmak Frrmack
- 86. Degree of liking is sensitive and vague Can’t we do something about this?
- 87. Degree of liking is sensitive and vague
- 88. “Enjoyment” from a movie is very high dimensional information
- 89. “Enjoyment” from a movie is very high dimensional information Rating means projecting this onto a single dimension
- 90. ?
- 91. But sometimes you just want to do the best projection you can What is my top twenty?
- 92. Degree of liking is sensitive and vague We cannot compare a movie with all others we've seen. We compare it to a limited set.
- 93. Trying to rate Star Wars
- 94. Trying to rate Star Wars
- 95. Trying to rate Star Wars 1 Map enjoyment to a specific scale
- 96. Trying to rate Star Wars 1 Map enjoyment to a specific scale
- 97. Trying to rate Star Wars 1 Map enjoyment to a specific scale
- 98. Trying to rate Star Wars 2 rating ose corresponding cho king for this degree of li
- 99. Trying to rate Star Wars But we cannot keep this entire history of enjoyment in mind
- 100. Trying to rate Star Wars But we cannot keep this entire history of enjoyment in mind We fuzzily remember a small subset
- 101. Trying to rate Star Wars But we cannot keep this entire history of enjoyment in mind We fuzzily remember a small subset We map based on this subset
- 102. Trying to rate Star Wars But we cannot keep this entire history of enjoyment in mind We fuzzily remember a small subset We map based on this subset
- 103. SAMPLING
- 104. BIASED SAMPLING
- 105. Tuesday
- 106. Tuesday
- 107. Friday
- 108. Friday
- 109. Degree of liking is sensitive and vague Can’t we do something about this?
- 110. We can certainly handle single comparisons ?
- 111. We can certainly handle single comparisons
- 112. We can certainly handle single comparisons less vague
- 113. We can certainly handle single comparisons little information
- 114. I can manually compare it with all others
- 115. And find exactly where it belongs Jo n e right after Indiana s ce ss ht before The Prin rig Bride
- 116. Full ranking: Compare all pairs
- 117. 1,000,000 comparisons? That’s a bit too much effort for me
- 118. We don’t need all of them
- 119. We don’t need all of them If
- 120. We don’t need all of them If ,
- 121. We don’t need all of them If , I have some information about
- 122. Compare a random sample of pairs
- 123. Use a ranking algorithm that utilizes all the information Good idea!
- 126. Elo rating system
- 127. Elo rating system
- 128. Elo rating system
- 130. Elo rating system -1.50 7.00 “hotness” range +1.50
- 135. Elo rating system -150 7.00 36% to win +150 -150 8.00 64% to win +150
- 136. Elo rating system How do we find out what these ranges are?
- 137. Elo rating system 5.00 5.00 5.00 5.00 5.00 5.00 Start with the same guess for every contender
- 140. Elo rating system 5.12 4.88 Update the best guesses accordingly
- 145. We don’t need all comparisons If , I have some information about
- 147. Elo rating system ? 7.61 4.02 89% to win 11% to win
- 148. Elo rating system 7.61 4.02 +.02 -.02 89% to win 11% to win
- 149. Elo rating system 7.61 4.02 -.53 +.53 89% to win 11% to win
- 150. Elo rating system 9.07 8.42 6.40 4.88 4.20 We now have scores on a single scale 3.03
- 151. Elo rating system 9.07 8.42 6.40 4.88 4.20 We now have scores on a single scale (estimates of people’s appreciation levels) 3.03
- 152. Elo rating system 9.07 1 8.42 2 6.40 3 4.88 4 and a ranking 3.03 4.20 5 6
- 153. Degree of liking is sensitive and vague Can we somehow apply this to movies, then?
- 154. We can do better
- 155. We can do better Bayesian ranking algorithms
- 156. We can do better Bayesian ranking algorithms Glicko (The Elo Killer) 1999
- 157. We can do better Bayesian ranking algorithms Glicko TrueSkill™ (The Elo Killer) 1999 2007
- 160. Degree of liking is sensitive and vague Liking (real time & remembered) depends on time and mood. Other people's opinions affect our own.
- 162. Bayesian ranking + - 4.01 4.46 82% to win + - 3% to draw 15% to win
- 163. Bayesian ranking ?
- 164. Bayesian ranking Elo: ? Best guess for the center 4.3
- 165. Bayesian ranking Bayesian: ? It could be centered around 4.3
- 166. Bayesian ranking Bayesian: ? It could also be centered around 4.2
- 168. Bayesian ranking Bayesian: ? Less likely but even around 4.5
- 171. Probability 2.0 2.5 3.0 3.5 4 4.5 5 Few comparisons: Lots of uncertainty (anything from 2.3 to 4.5 is quite possible)
- 172. Probability 2.0 2.5 3.0 3.5 4 4.5 5 After many comparisons: Quite sure (pretty much between 4.11 to 4.18)
- 173. Bayesian ranking ?
- 174. Bayesian ranking Lord of the Rings Star Wars 2.0 3.0 4.0 5.0
- 175. Bayesian ranking Lord of the Rings Star Wars 2.0 3.0 4.0 5.0
- 176. How did they do it? After: A small, constant increase in uncertainty before each comparison Probability Your rating changes with time. 3.5 4 4.5 5 uncertainty
- 177. Degree of liking is sensitive and vague Great! We have a system!
- 179. How many is too many? I don’t want to spend too much time on this
- 184. Minimum Effort Maximum Information Not reliable by itself Still carries a lot of information
- 187. What else can we do? I don’t want to spend too much time on this
- 189. Minimum Effort Maximum Information ? 98% to win 1% to draw 1% to win
- 190. Minimum Effort Maximum Information ? 98% Did not learn to win anything new
- 191. Minimum Effort Maximum Information ? Quite a bit of 2% new information to win
- 192. Minimum Effort Maximum Information ? I can calculate the expected amount of information from a comparison!
- 194. Minimum Effort Maximum Information Certain about both movies Won’t learn a lot
- 195. Minimum Effort Maximum Information Certain about both movies Won’t learn a lot
- 196. Minimum Effort Maximum Information Certain about both movies Won’t learn a lot Don’t know much about either Will learn a lot regardless of outcome
- 198. What are your top twenty movies? Irmak Frrmack
- 200. Quantifying human reactions are hard books celebrities songs tv shows food importance of issues politicans what to spend ‘fun’ budget on products teams in different sports
- 201. Degree of liking is sensitive and vague Amazing! Tuesday 3am Total garbage Sunday 12pm
- 202. Quantifying reactions is very useful
- 203. Quantifying reactions is very useful customized websites sorting search results recommendations connecting with other people of similar tastes identifying meaningful groups of similar products / people understanding your own preferences
- 204. Quantifying human reactions are hard Start with a rating, pose the correct comparisons
- 205. Quantifying human reactions are hard Start with a rating, pose the correct comparisons Every decision gets us closer
- 206. Degree of liking is sensitive and vague Amazing! Tuesday 3am Total garbage Sunday 12pm
- 207. Many comparisons for a movie over different days averages out mood and other factors
- 208. Many comparisons for a movie over different days averages out mood and other factors We can’t do much about social influence, but we should just accept that as natural part of how much we like things
- 209. Degree of liking is sensitive and vague Amazing! Tuesday 3am Total garbage Sunday 12pm
- 211. A great way of collecting desired data is to make it fun
- 213. Thanks
Notes de l'éditeur
- Social influence plays a big part in determining hits and misses