The document discusses recommender systems at Mendeley, an academic research platform. It describes how Mendeley uses recommender systems to help researchers organize their work, contextualize it within broader research, and connect with other researchers. The key components of Mendeley's recommender system are data sources, algorithms, business logic and analytics, and user interface. Mendeley's recommender system aims to provide personalized recommendations to researchers through different algorithms and lists of recommendations tailored to individual information needs.

1. Research recommendations at
Mendeley
Marco Rossetti
Data Scientist
@ross85
24/11/2015
Elsevier, Amsterdam

2. 2
Outline
Research recommendations at Mendeley
• What is Mendeley
• Recommender Systems at Mendeley
– Why
– Data Sources
– Algorithms
– Business Logic & Analytics
– User Interface
24/11/2015

3. 3
What is Mendeley
Research recommendations at Mendeley24/11/2015

4. 4
Mendeley builds tools to
help researchers …
Research recommendations at Mendeley
Read
&
Organize
Search
&
Discover
Collaborate
&
Network
Experiment
&
Synthesize
24/11/2015

5. 5
Read & Organize
Research recommendations at Mendeley
Reference
management
Cite-as-you-
write
Full-text article
search
Digitalised
annotations
24/11/2015

6. 6
Search & Discover
Research recommendations at Mendeley
Mendeley
Suggest
Literature
Search
Related
Documents
24/11/2015

7. 7
Collaborate & Network
Research recommendations at Mendeley
Research
network
Groups
24/11/2015

8. 8
Mendeley & Elsevier
Research recommendations at Mendeley24/11/2015

9. 9
Recommender Systems
Research recommendations at Mendeley24/11/2015

10. 10
Why Recommender Systems
at Mendeley?
Research recommendations at Mendeley
Vision:
“To build a personalised research advisor that helps
you to organise your work, contextualise it within the
global body of research, and connect you with
relevant researchers and artifacts.”
24/11/2015

11. 11
Recommender Systems
at Mendeley – Related Documents
Research recommendations at Mendeley24/11/2015

12. 12
Recommender Systems
at Mendeley – Mendeley Suggest
Research recommendations at Mendeley
https://www.mendeley.com/suggest/
24/11/2015

13. 13
Recommender System
Components
Research recommendations at Mendeley
Algorithms
Business Logic
and Analytics
User Experience
Data Sources Algorithms
Business
Logic
&
Analytics
User
Interface
24/11/2015

14. 14
• Mendeley
– User Libraries
• What the users have in their libraries (what they read, what they
annotate, what they highlight, what folders they have, etc. etc.)
– Articles metadata (title, authors, abstract, keywords, tags, etc. etc.)
– Groups
• Scopus
– Citation network
• Science Direct
– Logs
• …
Data Sources
Research recommendations at Mendeley24/11/2015

15. 15
Algorithms
Research recommendations at Mendeley
1. Collaborative filtering
User-based
If Alice read X, Y, Z and Bob read X, Y, Z and W, we recommend W to
Alice
+ Efficient for us because users << items
- Only for users with enough articles in the
library
Item-based
Users who read X also read Y
+ Item-item similarity matrix is useful to model last n articles read
- Expensive in our setting (millions of items)
24/11/2015

16. 16
Algorithms [2]
Research recommendations at Mendeley
1. Collaborative filtering (still)
Matrix factorization
+ Best CF model in literature
- A lot of latent factors, generate recommendations on a catalog of
million of items is too slow
1 1 1
1 1 1
? ? 1 ? 1 ?
1 1 1
1 1
1 1 1
U
n x k
V
k x m
X
n x m
X
≈
24/11/2015

17. 17
Algorithms [3]
Research recommendations at Mendeley
2. Content-based
I read articles about text mining, show me other stuff about text mining
+ Good for semi-cold users (users with only a few articles)
- Overspecialisation: items recommended are too similar
3. Popularity/Trending
I work in Computer Science, show me popular/trending
articles in Computer Science
+ Perfect for cold users
- Non personalised, discipline too broad
24/11/2015

18. 18
Algorithms [4]
Research recommendations at Mendeley
4. Citation Network
Articles similar to articles I cited
Articles that cite me
Articles from my co-author
+ Good for some kind of users
- Young researchers do not have (enough) publications
24/11/2015

19. 19
Offline experiments
Research recommendations at Mendeley
Offline Evaluation of 100+ algorithms variations on an
historical dataset
• Split data into training and testing based on timestamps: train until day
X, try to predict what users will add in the next day/week/month
• Computed different metrics to measure different dimensions:
• Accuracy (precision, recall, f-score, nDCG, MAP)
• Diversity
• Recency
• Popularity
• Consistency
• Coverage
24/11/2015

20. 20
Offline results
Research recommendations at Mendeley
Warm Users
Cold Users
24/11/2015
User Based CF
Item Based CF
Content Based
Citation Network
Popularity
Trending
Content Based
Popularity
Trending

21. 21
Business Logic / Analytics
Research recommendations at Mendeley
• Business put some constraints that could have an
impact on the recommendation experience
– Don’t show articles outside the user discipline
– Show articles only with a minimum readership
– Show only recommendations that you can explain (especially for people
recommendations, a different matter)
• Analytics
– Dashboard on the recommender statistics:
• Number of recommendations served
• Number of users with recommendations
• …
24/11/2015

22. 22
User Interface
Research recommendations at Mendeley
• Original idea: One list fits
all
Create a single list with the
best recommendations for
the user: use advanced
methods to take into
account every signal and
provide what is best for you!
24/11/2015

23. 23
User Interface [2]
Research recommendations at Mendeley
• However…
– Different kinds of users can have different information
needs!
– The same user in different contexts can have different
information needs!
VS
24/11/2015

24. 24
User Interface [3]
Research recommendations at Mendeley
• Solution: different lists!
• Provide multiple lists that satisfy different information needs
• More likely for a user to find something he is interested in
24/11/2015

25. 25
Online Survey
Research recommendations at Mendeley
Survey with Mendeley Advisors (pre-launch)
24/11/2015
Based on all the articles
Good
Bad
Popular
Good
Bad
Based on the last article
Good
Bad
Trending
Good
Bad

26. 26
Online Statistics
Research recommendations at Mendeley
Different statistics collected:
• overall and list
• click on title or
add to library
• different metrics:
– # users
– CR
– CTR
24/11/2015

27. 27
What’s next
Research recommendations at Mendeley
• New lists!
– Based on your research interests
– …
• Improve current lists
• Researchers you may want to follow
24/11/2015

28. 28Research recommendations at Mendeley24/11/2015
Thank you!