This document discusses modern perspectives on recommender systems and their applications at Mendeley. It covers the evolution of recommender problems from rating prediction to context-aware recommendations. It also discusses common recommender algorithms like collaborative filtering, content-based filtering and hybrid approaches. The document concludes by discussing how Mendeley uses recommenders for related research, researchers to follow, and other use cases.

1. Modern Perspectives on
Recommender Systems and their
Applications in Mendeley
Kris Jack and Maya Hristakeva
16/12/2014

2. Kris Jack, Chief Data Scientist
http://www.mendeley.com/profiles/kris-jack/
Maya Hristakeva, Senior Data Scientist
http://www.mendeley.com/profiles/maya-hristakeva/
Phil Gooch, Senior Data Scientist
http://www.mendeley.com/profiles/phil-gooch/

3. Overview
• The what and why of recommenders
• Evolution of the recommender problem
• Recommender algorithms
• Evaluating a recommender
• Recommender systems @ Mendeley

4. Overview
• The what and why of recommenders
• Evolution of the recommender problem
• Recommender algorithms
• Evaluating a recommender
• Recommender systems @ Mendeley

5. What is a recommender?
A recommendation system (recommender) is a push system that presents
users with the most relevant content for their context and needs
• helps users to deal with information overload
• recommenders are complementary to search
search engine
pull
recommendation engine
push
Information
Retrieval Information
request
infers context
and needs
Filtering

6. Recommenders @ Linkedin
50% of LinkedIn connections are from recommendations

7. Recommenders @ Linkedin

8. Recommenders @ Netflix
Stop 1% of users from cancelling subscription = $500M/year
Netflix invests $150M/year (300 people) in their content rec team

9. Recommenders @ ResearchGate

10. Why recommenders?
• Search and recommendations are complementary, have arms and legs!
• Higher usability, user satisfaction and engagement
• Increase product stickiness
• Monetise them
...and in the context of research...
Help researchers keep up-to-date with latest research, connect with
researchers in their field, contextualise their work within the global body of
research (articles, researchers, conferences, research groups, etc.)

11. Overview
• The what and why of recommenders
• Evolution of the recommender problem
• Recommender algorithms
• Evaluating a recommender
• Recommender systems @ Mendeley

12. Evolution of recommender problem
Problem: We have a massive collection of items (e.g. > 1 million).
We want to recommend 5 items that the user will like.

13. Evolution of recommender problem
First, seen as a ratings prediction problem. So, given some knowledge of the
user, estimate how much they will appreciate each item on scale of 1-5.
4.9
choose top 5 items with highest predicted ratings
4.7 4.7 4.6 4.5

14. Evolution of recommender problem
But do predicted ratings give the best order? Improve the recommender by
reranking a selection of items with high predicted ratings.
rerank items that are highly predicted
4.7 4.9 4.6 4.6 4.8

15. Evolution of recommender problem
Let’s improve the recommendations by optimizing the page in which they
appear.
deliver them in style

16. Evolution of recommender problem
Take the user’s context into account.
new to this
topic?
no yes

17. Evolution of recommender problem
Actively researching how to take other properties into account in context:
trustworthiness; freshness; diversity; serendipity; novelty; recency.
no at work? yes

18. Evolution of recommender problem
Rating prediction
Reranking
Page optimisation
Context-aware
Future: trustworthiness; freshness; diversity; serendipity; novelty; recency.
How to make recommendations?
On to the algorithms...
time

19. Overview
• The what and why of recommenders
• Evolution of the recommender problem
• Recommender algorithms
• Evaluating a recommender
• Recommender systems @ Mendeley

20. Recommender algorithms
A recommender processes information and transforms it into actionable
knowledge. Here we’ll focus on the algorithms that make this possible.
information flow (components often built in parallel)

21. Recommender algorithms
• Collaborative filtering (similarity and model-based)
• Content-based filtering
• Hybrid
• Non-traditional

22. Collaborative filtering
• User-based CF finds users who have similar appreciations for items as
you and recommends new items based on what they like.
• Item-based CF finds items that are similar to the ones you like.
Similarity is based on item cooccurrences (e.g. the users who bought x
also bought y).
Formal representation
Similarity-based CF
• ti: rating of user xi for item yi.
• Infer prediction function

23. Collaborative filtering
• Matrix Factorisation (SVD++)
• Clustering (K-means to LDA)
• LSH (Locality sensitive hashing)
• Restricted Boltzmann Machines
Formal representation of MF
• X: user-item ratings matrix
• U: user-latent factors matrix
• S: latent factor diagonal matrix
• V: latent factor-item matrix
Model-based CF

24. Collaborative filtering
Pros
• Minimal domain knowledge
required
• User and item features are not
required
• Produces good enough results
in most cases
Cons
• Cold start problem
• Requires high user:item ratio (1:
10)
• Needs standardised products
• Popularity bias (doesn’t play
well with the long tail)
• User-based CF
• Item-based CF
• Model-based CF

25. Content-based filtering
• Determine item similarity based on item content not usage data
• Recommend items similar to those that a user is known to like
• The user model:
• explicitly provided features/keywords of interest
• can be a classifier (e.g Naive Bayes, SVM, Decision trees)
Formal representation
• ti: rating of user xi for item yi, where xi and yi are feature vectors
• Infer prediction function

26. Content-based filtering
• Determine item similarity based on item content not usage data
• Recommend items similar to those that a user is known to like
• The user model:
• explicitly provided features/keywords of interest
• can be a classifier (e.g Naive Bayes, SVM, Decision trees)
Pros
• No cold start problem
• No need for usage data
• No popularity bias, can
recommend items with rare
features
Cons
• Item content needs to be
machine readable and meaningful
• Easy to pigeonhole the user
• Difficult to implement serendipity
• Difficult to combine multiple item’
s features together

27. Hybrid approaches
Method Description
Weighted Outputs from several techniques (in the form of scores or votes) are combined
with different degrees of importance to offer final recommendations
Switching Depending on situation, the system changes from one technique to another
Mixed Recommendations from several techniques are presented at the same time
Feature combination Features from different recommendation sources are combined as input to a
single technique
Cascade The output from one technique is used as input of another that refines the
result
Feature augmentation The output from one technique is used as input features to another
Meta-level The model learned by one recommender is used as input to another

28. Hybrid approaches
Combining user and item features and usage to benefit from both
Pros
• Often outperforms CF and CB
alone
Cons
• Can be a lot of work to get the
right balance

29. Non-traditional approaches
• Deep learning
• Social recommendations
• Learning to rank
• ...
Pros Cons
• Good for eking out those final
performance percentage points
• You can say you’re working with
current edge approaches ;)
• Less well understood
• Less supported in
recommendation toolkits
• Not recommended approaches
for your first recommender

30. Algorithms
• Typically employ collaborative filtering
• May need to use content-based filtering particularly to bootstrap
• Go advanced with a hybrid
• Do all of that before getting adventurous with state-of-the-art
Is your recommender doing well?
You don’t really know unless you evaluate it...

31. Overview
• The what and why of recommenders
• Evolution of the recommender problem
• Recommender algorithms
• Evaluating a recommender
• Recommender systems @ Mendeley

32. Evaluating a recommender
• Offline testing
• Online testing (A/B testing)

33. Offline testing
• Test offline before deploying
• Parameter sweep is quick
• Doesn’t offend real users
• n-fold cross validation:
• Take the users, items and
relationships between them
(e.g. clicked on, bought)
• Split into n folds, for training
(n-1) and testing (1)
• Attempt to predict the testing
data based on the training
data
• Popularity as baseline
Metrics
• Precision, recall and f-measure
• Receiver operating characteristic
(ROC) curve
• Normalised discounted cumulative
gain (NDCG)
• Mean reciprocal rank (MRR)
• Fraction of Concordant Pairs (FCP)
• ...

34. Online testing
• Offline performance isn’t a very
precise indicator
• Offline test is good sanity
check
• Online test gives real
performance
• A/B testing
• Deploy your systems that
perform ‘well enough’
• Compare them with each
other in real world
• Mind the pitfalls
Metrics
• The offline metrics +
• Conversion rate
• Open, view, click through rates
• Usage data (e.g. reordered item,
completed reading book)
• Hard to evaluate: trustworthiness;
freshness; diversity; serendipity;
novelty; recency.

35. Evaluating a recommender
• Start with offline testing
• Perform A/B testing but be aware of the common pitfalls
• Hard to evaluate performance in terms of: trustworthiness; freshness;
diversity; serendipity; novelty; recency.
How do we use recommenders?
On to a few of our use cases...

36. Overview
• The what and why of recommenders
• Evolution of the recommender problem
• Recommender algorithms
• Evaluating a recommender
• Recommender systems @ Mendeley

37. Recommenders @ Mendeley

38. Recommenders @ Mendeley
Related research for an article

39. Recommenders @ Mendeley
Related research for multiple articles

40. Recommenders @ Mendeley
Mendeley Suggest - personalised batch of recommended reading

41. Recommenders @ Mendeley
Researchers to follow on Mendeley

42. Recommenders @ Mendeley
Interesting activity from your social network

43. Recommenders @ Mendeley
• Recommenders are employed for a number of use cases
• Recommenders deliver different kinds of value depending upon use case
• Can reuse the same underlying recommender system and framework for all

44. Conclusions
• Recommenders are complementary to search and becoming mainstream
• although arguably can cater for a wider range of use cases
• When building a recommender, it’s common to predict ratings, rerank,
optimise the page and then introduce context-awareness
• In building a recommender, start with collaborative filtering if you can,
content-based if you need to bootstrap and then explore hybrids
• Open research questions remain as recommenders are used to tackle
trustworthiness; freshness; diversity; serendipity; novelty; recency

45. References
• Xavier Amatriain, The Recommender Problem Revisited (http://www.
slideshare.net/xamat/recsys-2014-tutorial-the-recommender-problem-revisited)
• Rec Sys 2014 (http://recsys.acm.org/recsys14/)

46. Thank you
www.mendeley.com