Watch talk ➟ http://bit.ly/1SGlQqj Interested in what your customers (or their customers) are talking about, and how that’s changing over time? The most popular 'topic trending' analyses depend on the investigation of how keyword (or term) usage change over time. Using keyword trends to index topic trends is highly suitable for short-form text, such as search terms, hashtags, or tweets. However, exploring trends in topic prevalence across longer, more free-form texts, such as call center telephone transcripts, is better served by grouping together topically-related words that co-occur. In this talk, we'll show an example of using Latent Dirichlet Allocation (LDA; Blei, Ng, & Jordan, 2003) in R to group words in call center transcripts into multiword topics over which we explore trends. We’ll also demonstrate the how the use of Structural Topic Modeling (Roberts, Stewart, Tingley, & Airoldi, 2013) can aid in further investigation of how document-level covariates (in this case, additional call- or caller-based characteristics) can affect topic prevalence and topic trends.

1. DATA
SCIENCE
POP UP
AUSTIN
Using LDA and Structural Topic Modeling to
Explore Trending Topics in a Call Center
Jordana Heller
Data Scientist, Mattersight
jheller

3. DATA
SCIENCE
POP UP
AUSTIN
#datapopupaustin
April 13, 2016
Galvanize, Austin Campus

4. Lightning Talk:
Using LDA and Structural Topic Modeling to
Explore Trending Topics in a Call Center
Jordana Heller @jheller
Data Science Pop-up Austin, April 13, 2016

5. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
What We Do

6. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Our goal: Topic Trends
3/31/2016 4/30/2016 5/31/2016 6/30/2016 7/31/2016
Identifying contents and prevalence of multiword topics present in conversation in an unsupervised way
Unexpected Prevalence Critical Spikes Escalating Frequency

7. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Our goals, continued
Manageable number of topics
Track expected and unexpected topics
Go deep: Contextualize topic usage

8. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Short text: Keywords, hashtags, ngrams

9. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Long text: Could use predetermined topics
Image credit: IBM Watson Concept Insights

10. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Long text: Or discover themes
Image credit: Blei, 2012, Communications of the ACM
Latent Dirichlet Allocation (LDA) (Blei et al., 2003)

11. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Great! How about contextualizing trends?
• Where are topics trending?
• Structural Topic Modeling (Roberts et al., 2013)
– Instead of relying on post-hoc comparisons,
includes covariates in LDA model
• Specifies priors as GLMs
• Word distribution determined by topic, covariates,
topic-covariate interaction
– Authors’ implementation: R package stm (available
via CRAN; all code on GitHub!)

12. Ready to talk pipeline!

13. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Data Collection and Preprocessing
Read
Transcripts
Add Call-level
Covariates
Preprocess
text
• Collocations
• -Stop words
• Stem/completion
• -Low freq terms
Create Term-
Document
Matrix

14. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Topic Model Creation
Retrieve
last topic
model
• For comparison
Create
current
topic model
• Detect number
of topics, or
specify
Create
topic labels

15. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Topic Model Comparison
Inspect overall
topic prevalence
Compare overall
topic prevalence
across periods
• Topics change!
Measure change in
word probability
distributions for each
new topic wrt each
old topic
• Match new to closest
previous match
below change
threshold (otherwise
new topic)
• Evaluate trends!
Estimate and
inspect effects of
covariates
Compare effects
of covariates
across periods
•Output can be
interpreted similarly
to regression

16. Example results: Hotel reservations
Covariates: booking, caller distress

17. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Booking
convention, center, mind, worry, philadelphia, inventory New
Decreasing
Increasing
Hit: > 1% of words on call
assigned to a given topic

18. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Booking
school, college, graduate, medical, clinic
New
Decreasing
Increasing
Hit: > 1% of words on call
assigned to a given topic

19. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Booking
30% beach, balcony, ocean, view
New
Decreasing
Increasing
Hit: > 1% of words on call
assigned to a given topic

20. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Booking
10% back, next, receive, listen, cash future
New
Decreasing
Increasing
Hit: > 1% of words on call
assigned to a given topic

21. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Booking
back, minute, system, run, inconvenience
New
Decreasing
Increasing
Hit: > 1% of words on call
assigned to a given topic

22. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Booking
42% confirm, email, arrival, local
New
Decreasing
Increasing
Hit: > 1% of words on call
assigned to a given topic

23. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Caller Distress
New
Decreasing
Increasing
Distress: > 30 seconds of linguistically-
identified dissatisfaction or negative emotion

24. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Caller Distress
square, city, price, hotel, manhattan, central
New
Decreasing
Increasing
Distress: > 30 seconds of linguistically-
identified dissatisfaction or negative emotion

25. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Trend Contextualization: Caller Distress
12% online, website, cancel, purchase, advance
New
Decreasing
Increasing
Distress: > 30 seconds of linguistically-
identified dissatisfaction or negative emotion

26. Nice!

27. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Our goals, revisited
Manageable number of topics
Track expected and unexpected topics
Go deep: Contextualize topic usage

28. ©2016 Mattersight Corporation. Mattersight Restricted Confidential Information.
Topic trends using
structural topic models
Thank you!

29. DATA
SCIENCE
POP UP
AUSTIN
@datapopup
#datapopupaustin