Learn how to automatically summarize academic literature with the BigML platform, presented by Professor Nigel L. Williams, Reader in Project Management and Research Lead in the Organizations and Systems Management Subject Group at the University of Portsmouth, in The United Kingdom. *Machine Learning School for Business Schools 2021: Virtual Conference.

1. Dealing with Text:
How to Automatically
Summarize Academic
Literature
Dr Nigel L. Williams, PMP
#BigMLSchool

2. Overview
• Why review the literature?
• Identifying a research gap
• Creating research questions
• (no code)Sources of text data
• Academic Abstracts
• Social Media
• Other
• Topic Models for exploring the literature
• Topic Model Process
• Findings
• Next Steps
#BigMLSchool

3. Is the problem
sufficiently clear?
What is the assumed problem?
No
NO
Does
the evidence
support the
problem?
What is the evidence for the
assumed problem?
Is the solution
sufficiently clear?
No
What is the preferred solution?
What is the evidence for the
preferred solution?
Does the evidence
support the
problem?
No
Apply
Why Should I
review Academic
Literature?
• Am I answering the
Right Question?
• Am I answering the
Right Question in the
Right Way?
Barends, &
Rousseau,
(2018).
#BigMLSchool

4. Why BigML?
• Intuitive
• No Code
• Browser Based
• Powerful
#BigMLSchool

5. Research Gaps in Business and Management
Lehtiranta,Junnonen, Kärnä & Pekuri (2015).
Gaps in Existing Theory
Neglect: Previous research has ignored aspects of the problem
Application : No empirical research has been done in an area
Research Overview: No summary of existing knowledge on a
topic exists
#BigMLSchool

6. PICOC
• Understand the geographical, professional or organizational context
• Questions that make sense in your context might be too vague
• For example: Why are our processes slow? has many answers
• You may wish to know about: Process Design, Process Benchmarking
(compare our process to others) or Process Resilience (How well do our
processes respond to disruption)
• Formulate a PICOC
#BigMLSchool

7. PICOC
Category Key Question Description
Population Who? Type of entity (employee,
people, organization) who may
be affected by the outcome
Intervention What or how? Technique, factor, independent
variable
Comparison Compared to what? Alternative intervention, factor,
variable
Outcome What are you trying
to accomplish/
improve/ change?
Objective, purpose, goal,
dependent variable
Context In what type of
organization/
circumstances?
Type of organization, sector,
relevant contextual factors
#BigMLSchool

8. Example
• Initial Question: Is remote working effective for professionals?
• Vague
• Improve by taking into account
• Population: the effect may be different for Construction managers than for Data Analysts
• Intervention: Home working or remote working
• Comparison: the effect may be different for Agile/Holacratic than for traditional working
• Outcome: “Effective” needs to be defined as the effect on performance is possibly different from
the effect on employee satisfaction;
• Context: the effect may be different for a Business School than for a Hotel.
• If you establish your PICOC, you can to determine if academic evidence is applicable to
your context.
#BigMLSchool

9. Structuring your search
• Clear research terms
• Step 1: Determine the two most important terms of your PICOC
• In most cases Intervention (management technique, independent variable) and the
Outcome (objective, outcome measure, dependent variable) are most useful
• Step 2: Finding alternative and related terms
• In some cases, these terms won’t be enough so you will need to identify alternative
and related terms
#BigMLSchool

10. Value of Academic Literature
• Academic Literature can
• Confirm the cause of the problem
• Confirm the relationship between cause, problem and organizational issues
• Confirm that the evidence is generalizable to the organizational context (PICOC)
#BigMLSchool

11. Sources of Text Data(for people who don’t
code)
#BigMLSchool

12. Scopus (Academic)
#BigMLSchool

13. Web of Science (Academic)
#BigMLSchool

14. Connected Papers (Academic)
#BigMLSchool

15. Publish or Perish (Academic, Harzing.com)
#BigMLSchool

16. Netlytic (Twitter, YouTube, Reddit)
#BigMLSchool

17. Facepager (Facebook, Twitter, Reddit)
#BigMLSchool

18. Octoparse (Webscraper)
#BigMLSchool

19. Note
• Ensure that you have ethical approval from your institution
• Ensure that you are operating within the platform’s rules
#BigMLSchool

20. Why Topic Modelling
• Categorization of Text
• Options
• Manual
• Dictionary-based
• Supervised Machine Learning
• Unsupervised machine-learning (Topic Modelling)
#BigMLSchool

21. Advantages and Disadvantages of Topic
Modelling
• Advantages
• Automated
• Reproducible results
• Large scale/heterogenous text
• Disadvantage
• Researcher has to interpret categorizations
#BigMLSchool

22. Topic Modelling using LDA
• LDA assumes all documents
to be generated from a fixed
set of topics,
• Each document exhibits these
topics in different
proportions from 0 percent
(does not mention the topic)
to 100 percent (exclusively
discusses the topic).
Document1 Document2 Document3
Topic2
Topic1 Topic3
Word1 Word2 Word3
% %
%
%
%
%
Per- Document Topic
Distribution
Per-Topic Word Distribution
Blei, Ng & Jordan (2003). #BigMLSchool

23. Interpreting Topic Models
• Questions to ask (Boyd-Graber, Mimno, & Newman(2014) )
• Are individual topics meaningful, interpretable, coherent, and useful?
• Are assignments of topics to documents meaningful, appropriate, and
useful?
• Topic Modelling problems
• Too many common words or too many specific words (e.g., names)
• Mixed topics that contain more than one topic and should be split.
• Identical topics where two topics are proposed that are the same
• Nonsensical topic if documents have the same structural pattern
#BigMLSchool

24. BigML Topic Modelling Process
BigML
Dataset
BigML Data
Preprocessing
BigML Topic
Modeling
Experiments
Topic 1
Topic 0
Topic n
PICOC
(P: Customer,
I:Segmentation)
Scopus
Search for
Abstracts
#BigMLSchool

25. Preprocessing
• Stop-word elimination: removal of common words or words that are not useful for the
study
• Stemming: conversion of words into their root form
• Tokenizing: dividing a text input into tokens like phrases or words.
• Identifying word combinations such as bigram (2) or trigram (3) to be considered as a
single term
#BigMLSchool

26. Topic Modelling Experiments
Model Max no .
Of Topics
No of
Terms
Max n-
grams
Number
of words
per topic
Specifie
d words
excluded
Non
dictionary
words
excluded
Non-
language
characters
excluded
Numeric
digits
exclude
d
Single
tokens
exclude
d
1 Auto 4096 Trigram 10 YES YES YES YES YES
2 Auto 4096 Trigram 20 YES YES YES YES YES
3 Auto 2048 Trigram 20 YES YES YES YES YES
4 Auto 2048 Bigram 20 YES YES YES YES YES
5 Auto 1024 Trigram 20 YES YES YES YES YES
Hajiheydari, Talafidaryani,Khabiri, & Salehi (2019). #BigMLSchool

27. Other Settings
• Stop word removal: Aggressive
• Case sensitive off
• Sampling: Census
#BigMLSchool

28. Experiment 1 Results
#BigMLSchool

29. Too Many Specific words
Terms Probability
emerald publishing
limited 0.12694
publishing limited 0.12309
emerald publishing 0.12293
purchase intention 0.05056
marketing managers 0.02846
consumer behaviour 0.02802
learning outcomes 0.02368
teaching notes 0.02259
brand management 0.0193
practical implications 0.01752
• “Emerald Publishing” dominates topic
• Solution: Add “Emerald publishing” and
similar terms to exclusion list
#BigMLSchool

30. Experiment 2
#BigMLSchool

31. Findings
latent class 0.09261 tourism market 0.05738
logit model 0.03769 destination marketing 0.03723
consumer
preferences 0.0365 tourism destinations 0.03245
choice model 0.03125 tourism industry 0.02635
conjoint analysis 0.02823 tourists visiting 0.02204
class model 0.02605 cultural heritage 0.02043
latent class
model 0.02378 heritage site 0.01969
mixture model 0.02373 international tourists 0.01891
representative
sample 0.0111 cognizant comm 0.01345
finite mixture
model 0.01085 tourist market 0.01281
service
attributes 0.01075 sport tourists 0.01189
segment
membership 0.01001 destination choice 0.01162
#BigMLSchool

32. Experiment 3
#BigMLSchool

33. Experiment 3 Results
logit model 0.04901 data mining 0.06056 factor analysis 0.12444
choice model 0.03727 vector machine 0.05919 analysis revealed 0.04768
multinomial logit 0.02727 support vector machine0.05858
hierarchical
cluster 0.0382
multinomial logit model 0.01851 based approach 0.04339 analysis identified 0.03411
stated preference 0.01833 clustering algorithm 0.0401 step cluster 0.0316
market segmentation
analysis 0.01646 logistic regression 0.03648
step cluster
analysis 0.02531
mode choice 0.0159 decision tree 0.03572 exploratory factor 0.02447
price premium 0.01547 clustering techniques 0.02238
hierarchical
cluster analysis 0.02327
based market
segmentation 0.00839 feature selection 0.01191
cluster analysis
revealed 0.01179
optimization model 0.00783 mining techniques 0.01143 clusters based 0.01017
choice behaviour 0.00733 random forest 0.01123 cluster solution 0.00944
#BigMLSchool

34. Create Batch Topic Distribution
#BigMLSchool

35. Next Steps
• Export Dataset labelled with topics for manual examination
• Download academic papers in specific topics for manual or additional text
analysis/topic modelling
• Use Topic Model as an input for additional predictive models
#BigMLSchool

36. My Contacts
• Nigel.williams@port.ac.uk
• https://researchportal.port.ac.uk/portal/en/persons/nigel-williams
• https://www.linkedin.com/in/drnigelwilliams/
• www.ResponsiblePM.com
#BigMLSchool

37. Resources
• Barends, E., & Rousseau, D. M. (2018). Evidence-based management: How to use evidence to
make better organizational decisions. Kogan Page Publishers.
• Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. the Journal of machine
Learning research, 3, 993-1022.
• Boyd-Graber, J., Mimno, D., & Newman, D. (2014). Care and feeding of topic models: Problems,
diagnostics, and improvements. Handbook of mixed membership models and their
applications, 225255.
• Debortoli, S., Müller, O., Junglas, I., & vom Brocke, J. (2016). Text mining for information
systems researchers: An annotated topic modeling tutorial. Communications of the Association
for Information Systems, 39(1), 7.
• Hajiheydari, N., Talafidaryani, M., Khabiri, S., & Salehi, M. (2019). Business model analytics:
technically review business model research domain. foresight.
• Lehtiranta, L., Junnonen, J. M., Kärnä, S., & Pekuri, L. (2015). The constructive research
approach: Problem solving for complex projects. Designs, methods and practices for research
of project management, 95-106.
#BigMLSchool