While there have been several studies on how users experience algorithmic recommendations and their explanations, we know relatively little about human recommendations and which item aspects humans highlight when describing their own recommendation needs. A better understanding of human recommendation behavior could help us design better recommender systems that are more attuned to their users. In this paper, we take a step towards such understanding by analyzing a Reddit community dedicated to requesting and providing for recommendations: /r/ifyoulikeblank. After a general analysis of the community, we provide a more detailed analysis of the prevalent music requests and the example items used to ask for these recommendations. Finally, we compare these human recommendations to algorithmic recommendations to better char- acterize their differences. We conclude by discussing the implications of our work for recommender systems design.

1. "IF I LIKE _______, WHAT ELSE WILL I LIKE?":
ANALYZING A HUMAN RECOMMENDATION
COMMUNITY ON REDDIT
Thi Binh Minh Cao, Toine Bogers
iConference 2024
IT UNIVERSITY OF COPENHAGEN

2. IT UNIVERSITY OF COPENHAGEN
PART 1
TITLE
Introduction

3. ▸ Recommender systems help users make decisions by suggesting and presenting content in a relevant way
– Recent decades have seen a lot of work on human-recommender interaction (Alvarado et al., 2019; Freeman et al.,
2022, 2023; Lee et al., 2019) and explainable recommendations (Tintarev & Masthoff, 2007; Zhang et
al., 2020; Karimi et al., 2022)
▸ Very little work has focused on human recommendations
– What does recommendation between two individuals without any
algorithmic intervention look like?
– Which item attributes do human highlight when describing their
own recommendation needs or explaining recommendations to
others?
– A deeper understanding of human recommendation could help us
design better recommender systems that are more attuned to their
users
INTRODUCTION 3

4. ‘IF YOU LIKE BLANK’
▸ To answer this question, we analyze the Reddit subcommunity /r/ifyoulikeblank
– Purpose is to solicit and provide “recommendations of any relevant media— whether it be music,
television, video games, movies, or anything else”
• Users can request recommendations (“If I like the vibrant musical experimentation and catchiness of Hip Hop
acts like Kid Cudi and Kanye West, what other Hip Hop would I like?”)
• Users can provide recommendations (“If you like ‘Arrested Development’, you might like ‘Agents of
Cracked’.”)
▸ Quite popular for a relatively unknown subreddit
– Over 23,000 posts and 87,000 comments in 2022 alone
– Ranked in the top 1000 (#813) of most popular subreddits with ~944,000 members (April 30, 2023)
4

5. INTRODUCTION 6
What Else Will I Like
If I Like
If You Like

6. INTRODUCTION 7
Recommendation/information need
Item attributes
Seed examples

7. INTRODUCTION 8

8. RESEARCH QUESTIONS
▸ RQ1 What characterizes the human recommendations and recommendation needs
shared 0n /r/ifyoulikeblank?
▸ RQ2 What characterizes the music requests and the recommendations provided by
other users?
▸ RQ3 How do human music recommendations compare to those provided through
algorithmic means?
9

9. IT UNIVERSITY OF COPENHAGEN
PART 1
TITLE
Methodology

10. METHODOLOGY
▸ Data collection from
– /r/ifyoulikeblank contains two types of threads
• Threads requesting recommendations (IIL … WEWIL)
• Threads offering recommendations (IYL)
– Posting guidelines encourage including at least one example (max 9)
and a description of why they like it/them
– Received 57.0 posts/day and 209.7 comments/day (May 1, 2022 –
April 30, 2023)
11
Analyzing a Human
Fig. 1. Change in activity levels for
the /r/ifyoulikeblank subreddit.
three subreddits from June 1, 2018
threads. From this sample, we ra
initial post and all associated com
of these 1,920 posts, we have the
the rank of a post in the thread,
minus downvotes).
3.2 Data Annotation
The presence of the [IIL] and [IYL] t
an initial post was a recommenda
The remaining 43 posts were cat
Analyzing a Human Recommendation Commun
Fig. 1. Change in activity levels for (a) posts, (b) comments, an
the /r/ifyoulikeblank subreddit.
three subreddits from June 1, 2018 to August 3, 2018, resultin
threads. From this sample, we randomly selected 300 threa
initial post and all associated comment posts (n = 1,620) fo
of these 1,920 posts, we have the post title and text, the us
Analyzing a Human Recommendation Community on Reddit 5

11. METHODOLOGY
▸ Data collection
– Crawled 4,957 /r/ifyoulikeblank discussion threads from June 1, 2018 to August 3, 2018
– Randomly selected 300 threads consisting of the initial post and all associated comment
posts (n = 1,620) for analysis
• For each of these 1,920 posts, we have
★ Post title and text
★ User ID of the poster
★ Rank of a post in the thread
★ Post score (= the number of upvotes minus downvotes)
12

12. METHODOLOGY
▸ Data annotation
– Post type (request vs. offer)
• 257 of 300 posts included one of the [IIL] and [IYL] tags
• 43 posts were categorized manually
★ 1 post was spam and was filtered from the dataset
★ Final dataset for annotation and analysis contained 299 original posts and 1,620 comments
13

13. METHODOLOGY
– Open coding
• We developed an initial set of codes
based on the title and text of 50
random posts
– Axial coding
• Settled on final coding scheme
– Final coding
• Applied the coding scheme to the
299 original posts and 1,620
comments
14
6 T.B.M. Cao and T. Bogers
Table 1. Overview of the coding scheme.
Code Description
Seed item The seed item(s) in the initial posts for which recommen-
dations were requested or offered
Recommended item The recommended item(s) mentioned either in the initial
post or by other users in the comments
Recommendation
quality
Feedback from the original requester on the quality of a
recommendation, translated to a binary scale
Positive attributes Positive attributes of a recommended item according to
the user writing the post or comment
Negative attributes Negative attributes of a recommended item according to
the user writing the post or comment
Media type The type of media recommendations are requested or of-
fered for, further subdivided into six categories: (1) Music
(e.g., song, artist, album, genre), (2) Books (e.g., book,
author, manga, story); (3) Movies (e.g., movie, actor, di-
rector, producer); (4) TV shows (e.g., TV show, cartoon,
anime); (5) Games (e.g., game, gamer); and (6) Mis-
cellaneous (e.g., car, food, podcast, YouTube channel,
website).

14. IT UNIVERSITY OF COPENHAGEN
PART 1
TITLE
RQ1
Characteristics of requests

15. RQ1
▸ Characteristics of human recommendations and recommendation needs
– Post types
• Subreddit is overwhelmingly used for requests for recommendation (97% of the 299 posts)
• No significant difference in terms of thread length
★ 8.4 comments/thread for requests vs. 8.1 for offers
• Rest of the analysis focuses on request for recommendations
20

16. RQ1: POST LENGTH 21
Analyzing a Human Recommendation Community on Reddit 7
Mrequest = 41.3
Mreply = 24.3

17. RQ1: THREAD LENGTH 22
Analyzing a Human Recommendation Community on Reddit 7
Mreply_count = 6.4

18. RQ1: USER ACTIVITY 23
Analyzing a Human Recommendation Community on Reddit 7
Mcomments = 1.95

19. RQ1: COMMENTS 24
Analyzing a Human Recommendation Community on Reddit 7

20. RQ1: SEED ITEMS 25
Analyzing a Human Recommendation Community on Reddit 7
Mseed_items = 3.0

21. RQ1: RECOMMENDED ITEMS 26
Analyzing a Human Recommendation Community on Reddit 7
Mrecommendations = 12.1

22. RQ1: MEDIA TYPES 27
Analyzing a Human Recommendation Community on Reddit
Fig. 3. Distributions of (a) different top-level and lower-level media types over
requests, and (b) the top 10 genres attached to the 196 artist seeds mentioned
different music requests.
Analyzing a Human Recommendation Community on Reddit 9
Fig. 3. Distributions of (a) different top-level and lower-level media types over all 290
requests, and (b) the top 10 genres attached to the 196 artist seeds mentioned in 82
different music requests.
Table 2. Request activity levels split by media type. The numbers in the four right-most
columns are all calculated on a per-request basis, e.g., books receive 3.8 comments
per request on average of which 0.1 contain feedback from the OP, who receives 4.7
recommendations on average per request.
Media type n Length Comments Feedback Recommendations
Books 9 50.6 3.8 0.1 4.7
Games 9 43.9 5.6 0.2 9.6
Movies 18 43.0 9.3 0.4 19.2
Music 233 41.8 6.0 0.6 12.0
TV 18 24.3 7.2 0.3 14.2
Miscellaneous 10 42.9 4.5 0.1 6.3

23. Analyzing a Human Recommendation Community on Reddit
Fig. 3. Distributions of (a) different top-level and lower-level media types over
requests, and (b) the top 10 genres attached to the 196 artist seeds mentioned
different music requests.
RQ1: MEDIA TYPES 28
Analyzing a Human Recommendation Community on Reddit 9
Fig. 3. Distributions of (a) different top-level and lower-level media types over all 290
requests, and (b) the top 10 genres attached to the 196 artist seeds mentioned in 82
different music requests.
Table 2. Request activity levels split by media type. The numbers in the four right-most
columns are all calculated on a per-request basis, e.g., books receive 3.8 comments
per request on average of which 0.1 contain feedback from the OP, who receives 4.7
recommendations on average per request.
Media type n Length Comments Feedback Recommendations
Books 9 50.6 3.8 0.1 4.7
Games 9 43.9 5.6 0.2 9.6
Movies 18 43.0 9.3 0.4 19.2
Music 233 41.8 6.0 0.6 12.0
TV 18 24.3 7.2 0.3 14.2
Miscellaneous 10 42.9 4.5 0.1 6.3

24. RQ2
Characteristics of music requests

25. RQ2: MUSIC ELEMENTS & GENRES 30
Analyzing a Human Recommendation Community on Reddit 9
Fig. 3. Distributions of (a) different top-level and lower-level media types over all 290

26. Analyzing a Human Recommendation Community on Reddit 9
Fig. 3. Distributions of (a) different top-level and lower-level media types over all 290
RESULTS (RQ1) 31

27. RQ2: POSITIVE & NEGATIVE ATTRIBUTES
▸ How do people describe their seed items and recommendations?
– 1,462 music requests and comments were annotated for positive and negative description
– Users rarely describe items beyond simply posting the song and/or artist
• 55 posts contained positive descriptions and 13 posts contained negative attributes
• Positive descriptions describe what users like about a song (e.g., “scifi vibe”, “weirdly chipper stoner
rock”, “upbeat music”)
• Negative comments describe what is lacking from a recommendation (e.g., “lacking positive
atmosphere”, “chorus not powerful enough”, “lacking cool element”)
32

28. RQ3
Human vs. algorithmic recommendations

29. RQ3: HUMAN VS. ALGORITHMIC RECOMMENDATIONS 34
Analyzing a Human Recommendation Community
Fig. 4. Distributions of popularity scores for seed songs versus the hu
▸ Is there overlap between human and algorithmic
recommendations?
– We request 10 recommendations for each seed song from
Spotify
– Very little overlap between human and algorithmic top-10
lists (Jaccard overlap of 0.000281)
▸ Does popularity bias exist in the seed items and
recommendations?
– We collected popularity scores for each song from Spotify
– Spotify recommendations have significantly lower
popularity on average than human recommendation

30. IT UNIVERSITY OF COPENHAGEN
PART 1
TITLE
Discussion

31. DISCUSSION
▸ Takeaways
– Users that provide more example items and links to those examples when describing their recommendation
need are more successful
– There are hints of media type having an influence on success
– Human (music) recommendations exhibit a greater popularity bias than algorithmic ones
▸ Implications
– Informing the design of item-to-item recommender algorithms
– Training material for conversational recommenders
– Informing the design of music recommendation interfaces for need explication
– Designing domain-specific information extraction tools
36

32. DISCUSSION
▸ Limitations
– No valid conclusions to be drawn on any domain other than music
▸ Future work
– More thorough analysis of the textual feedback that users provide on recommendation
– In-depth analysis of the (dis)similarities between the human and Spotify recommendations
37

33. QUESTIONS?

34. REFERENCES
▸ Alvarado, O., Abeele, V.V., Geerts, D., Verbert, K.: “I really don’t know what ‘Thumbs up’ means”: Algorithmic
Experience in Movie Recommender Algorithms. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M.,
Zaphiris, P. (eds.) INTERACT 2019: Proceedings of the 2019 Conference on Human-Computer Interaction. pp. 521–541.
Springer (2019)
▸ Freeman, S., Gibbs, M., Nansen, B.: ‘Don’t mess with my algorithm’: Exploring the Relationship between
Listeners and Automated Curation and Recommendation on Music Streaming Services. First Monday 27(1) (2022)
▸ Freeman, S., Gibbs, M., Nansen, B.: Personalised But Impersonal: Listeners’ Experiences of Algorithmic Curation
on Music Streaming Services. In: CHI ’23: Proceedings of the 2023 CHI Conference on Human Factors in Computing
Systems. ACM, New York, NY, USA (2023). https://doi.org/10.1145/3544548.3581492
▸ Karimi, A.H., Barthe, G., Schölkopf, B., Valera, I.: A Survey of Algorithmic Recourse: Contrastive Explanations and
Consequential Recommendations. ACM Computing Surveys 55(5), 1–29 (2022)
▸ Lee, J.H., Pritchard, L., Hubbles, C.: Can We Listen To It Together?: Factors Influencing Reception of Music
Recommendations and Post-Recommendation Behavior. In: ISMIR ’19: Proceedings of the 20th International
Society for Music Information Retrieval Conference. pp. 663–669 (2019)
39

35. REFERENCES
▸ Tintarev, N., Masthoff, J.: A Survey of Explanations in Recommender Systems. In: Proceedings of the 2007 IEEE
23rd International Conference on Data Engineering Workshop. pp. 801–810. IEEE (2007)
▸ Zhang, Y., Chen, X., et al.: Explainable Recommendation: A Survey and New Perspectives. Foundations and
Trends in Information Retrieval 14(1), 1–101 (2020)
40