Mendeley Talk

1. What can we learn from topic
modeling on 350M documents?
William Gunn
Head of Academic Outreach
Mendeley
@mrgunn – https://orcid.org/0000-0002-3555-2054

2. Based in London, Mendeley is
researchers, graduates and software
developers from...

3. The opposite problem
 We have the papers (400M) and are
looking for the best way to turn
them into structured knowledge.
 We have useful triage indicators -
#altmetrics, reproducibility
 You have great use cases

4. ...and aggregates
data in the cloud
Mendeley
extracts
research data…
Collecting rich signals
from domain experts.

5. Rich user profile data

6. TEAM Project
academic knowledge management solutions
• Algorithms to determine the content similarity of academic papers
• Performing text disambiguation and entity recognition to
differentiate between and relate similar in-text entities and authors
of research papers.
• Developing semantic technologies and semantic web languages with
the focus of metadata integration/validation
• Investigate profiling and user analysis technologies, e.g. based on
search logs and document interaction.
• We will also improve folksonomies and through that, ontologies of
text.
• Finally, tagging behaviour will be analysed to improve tag
recommendations and strategies.
• http://team-project.tugraz.at/blog/

7. Semantics vs. Syntax
• Language expresses semantics via syntax
• Syntax is all a computer sees in a research
article.
• How do we get to semantics?
•Topic Modeling!

8. Distribution of Topics
35%
30%
25%
20%
15%
10%
5%
0%
Bio Phys Engineer Comp
Sci
Psych &
Edu
Business Law Other

9. Subcategories of Comp. Sci.
20%
15%
10%
5%
0%
AI HCI Info Sci Software
Eng
Networks

11. Generated topics – Comp. Sci.

12. Generated Topics - Biology

13. Categorization is imperfect

14. Categorization As A Process
Thing
Process
Reaction
Catalysis
Enzymatic

15. Categorization As A Process
Thing
Process
Reaction
Catalysis
Enzymatic

16. Categories change over time

17. Can we assist triage?

18. Code Project
Use case = mining research papers for facts
to add to LOD repositories and light-weight
ontologies.
• Crowd-sourcing enabled semantic enrichment & integration
techniques for integrating facts contained in unstructured
information into the LOD cloud
• Federated, provenance-enabled querying methods for fact
discovery in LOD repositories
• Web-based visual analysis interfaces to support human based
analysis, integration and organisation of facts
• Socio-economic factors – roles, revenue-models and value
chains – realisable in the envisioned ecosystem.
• http://code-research.eu/

22. Metrics as a discovery tool

23. We didn ’t
see that a target is
more likely to be validated if it
was reported in ten publications
or in two publications
NATURE REVIEWS DRUG DISCOVERY 10, 712 (SEPTEMBER 2011)

24. Either the results were reproducible
and showed transferability in other
models, or even a 1:1 reproduction of
published experimental procedures
revealed inconsistencies between
published and in-house data
NATURE REVIEWS DRUG DISCOVERY 10, 712 (SEPTEMBER 2011)

25. There is no Gold Standard
 Amgen: 47 of 53 “landmark” oncology publications could
not be reproduced.
 Bayer: 43 of 67 oncology & cardiovascular projects were
based on contradictory results
 Dr. John Ioannidis: 432 publications purporting sex
differences in hypertension, multiple sclerosis, or lung
cancer. Only one data set was reproducible.

26. Building a reproducibility dataset
• Mendeley and Science Exchange have
started the Reproducibility Initiative
• working with Figshare & PLOS to host data
& replication reports
• building open datasets backing high-impact
work
• extending the “executable paper” concept
to biomedical research

27. Make it porous & part of the
web.
 Our success as a crowdsourcing platform
is largely due to our openness & end-user
usefulness.
 Communities must be open if they are to
thrive.

28. www.mendeley.com
william.gunn@mendeley.com
@mrgunn