Looking at text clustering using probabilistic methods (LDA) and correlating with structured data, in particular geolocation

1. Web Science & Technologies
University of Koblenz ▪ Landau, Germany
Topic Discovery in Unstructured Data:
The Next Generation
Christoph Kling, Sergej Sizov, Steffen Staab

2. Understanding Social Media: Example Yahoo News Comments
• Many
comments
• More opinions
• Commenting
different
(sub)topics
WeST Steffen Staab Topic Detection - TNG
2 of 25 10/09/2012 2

3. Discovering topics using LDA
WeST Steffen Staab Topic Detection - TNG
3 of 25

4. Browse by topic
more..
more..
WeST Steffen Staab Topic Detection - TNG
4 of 25

5. We have: Topic-Document – All Fine?
How do we understand We work on:
the topics? • Opinions about topics
Are all topics of same • Diversity of opinions
value? • Localisation of topics
Is there structured data to • Time-varying topic
correlate? models (Blei, Lafferty)
• Space • ....
• Time • Geo-varying topic
• Network information models
WeST Steffen Staab Topic Detection - TNG
5 of 25

6. Geo-located social media content
BMW
Audi Audi
Citroen
BMW Chevrolet Peugeot
Renault
Citroen
Chevrolet
BMW
Pontiac Mercedes Audi
Chevrolet
Pontiac
Fiat
Mercedes
BMW
WeST Steffen Staab Topic Detection - TNG
6 of 25

7. Geo-located social media content
chevrolet
citroen
pontiac renault
bmw BMW
Audi peugeot
mercedes Audi
Citroen bmw
audi Chevrolet Peugeot
BMW
Renault
Citroen
Chevrolet
BMW
Pontiac Mercedes Audi
Chevrolet
Pontiac
Fiat
Mercedes
BMW
bmw
audi
mercedes
fiat
citroen
WeST Steffen Staab Topic Detection - TNG
7 of 25

8. Related work
chevrolet
citroen
pontiac renault
bmw BMW
Audi peugeot
mercedes Audi
Citroen bmw
audi Chevrolet Peugeot
BMW
Renault
Citroen
Chevrolet
BMW
Pontiac Mercedes Audi
Chevrolet
Pontiac
Fiat
Mercedes
BMW
bmw
audi
mercedes
fiat
citroen
LGTA, Yin et al. 2011
WeST Steffen Staab Topic Detection - TNG
8 of 25

9. Problem
Geographical distribution of topics
Language areas Dominating religion
WeST Steffen Staab Topic Detection - TNG
9 of 25

10. Our approach
chevrolet
citroen
pontiac renault
bmw BMW
Audi peugeot
mercedes Audi
Citroen bmw
audi Chevrolet Peugeot
BMW
Renault
Citroen
Chevrolet
BMW
Pontiac Mercedes Audi
Chevrolet
Pontiac
Fiat
Mercedes
BMW
bmw
audi
mercedes
fiat
citroen
WeST Steffen Staab Topic Detection - TNG
10 of 25

11. Our approach
chevrolet citroen
pontiac renault
BMW
bmw Audi
Audi peugeot
mercedes Citroen bmw
audi BMW Chevrolet Peugeot
Renault
Chevrolet Citroen
BMW
Pontiac Mercedes Audi
Chevrolet
Pontiac chevrolet
Fiat citroen
pontiac Mercedes renault
bmw BMW
BMW bmw peugeot
Audi Audi
mercedes audi Citroen bmw
audi Chevrolet mercedes Peugeot
BMW
fiat
Renault
citroen Citroen
Chevrolet
BMW
Pontiac Mercedes Audi
Chevrolet
Pontiac
Fiat
Mercedes
BMW
bmw
audi
mercedes
WeST Steffen Staab Topic Detection - TNG fiat
citroen
11 of 25

12. Geographical network construction
Data points Spatial region centroids Geographical network
WeST Steffen Staab Topic Detection - TNG
12 of 25 10/09/2012 12

13. Topic detection
Topic assignments
WeST Steffen Staab Topic Detection - TNG
13 of 25

14. Topic detection
Topic assignments
WeST Steffen Staab Topic Detection - TNG
14 of 25

15. Topic detection
Topic assignments
WeST Steffen Staab Topic Detection - TNG
15 of 25

16. Topic detection
Topic assignments
WeST Steffen Staab Topic Detection - TNG
16 of 25

17. Topic detection
Topic assignments
WeST Steffen Staab Topic Detection - TNG
17 of 25

18. Topic detection
Topic exchange between adjacent clusters:
Pontiac
Chevrolet
BMW BMW
Pontiac
Chevrolet
WeST Steffen Staab Topic Detection - TNG
18 of 25

19. Topic detection
Topic exchange between adjacent clusters:
spatial region A
spatial region B
Pontiac spatial region
B D
A
Chevrolet
1 D
BMW C BMW
Pontiac
Chevrolet
document1
spatial region C
WeST Steffen Staab Topic Detection - TNG
19 of 25

20. Topic detection
Topic exchange between adjacent clusters:
spatial region A
spatial region B
Pontiac spatial region
B D
A
Chevrolet
1 D
BMW C BMW
Pontiac
Chevrolet
document1
spatial region C
WeST Steffen Staab Topic Detection - TNG
20 of 25

21. Topic detection
Pontiac
B
A Chevrolet
1 D
BMW C BMW
Pontiac
Chevrolet
A B
1 1 C
1
is drawn from
with equal probability
WeST Steffen Staab Topic Detection - TNG
21 of 25

22. Visualisation
chevrolet 0.35 bmw 0.29
bmw 0.18 audi 0.18
cadillac 0.16 fiat 0.10
pontiac 0.09 citroen 0.09
gmc 0.07 renault 0.09
buick 0.06 peugeot 0.08
audi 0.05 mercedesbenz 0.06
chevrolet 0.05
WeST Steffen Staab Topic Detection - TNG
22 of 25

23. Visualisation
bmw 0.63 fiat 0.66 renault pontiac 0.92
mercedesbenz 0.17 bmw 0.10 0.28
audi 0.13 citroen 0.09 citroen
renault 0.05 0.22
peugeot
0.15
bmw
0.10
audi
0.09
fiat
0.07
WeST Steffen Staab Topic Detection - TNG
23 of 25

24. Topic Detection: The next generation
GeoMTD
• Better understandability: „nicer regions“
• Improved quality
• Better explanation of the data
• Measured in terms of reduced perplexity
• about half compared to related work
WeST Steffen Staab Topic Detection - TNG
24 of 25

25. Topic Detection: The next generation
Other next generation mechanisms for understanding social
media:
• Opinions
• adding vocabularies with meaning (LIWC, POMS,...)
• Diversity
• maximizing for spread of topics and opinions
• Author-topic-time...
Need to balance between complexity of
model and sparsity of data!
WeST Steffen Staab Topic Detection - TNG
25 of 25

26. Web Science & Technologies
University of Koblenz ▪ Landau, Germany
Thank you for your attention!

27. References
Hierarchical Dirichlet processes
by: Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei
In: Journal of the American Statistical Association, Vol. 101 (2006) , p. 1566-1581.
GeoFolk: latent spatial semantics in web 2.0 social media.
by: Sergej Sizov
In: WSDM ACM (2010) , p. 281-290.
Geographical topic discovery and comparison.
by: Zhijun Yin, Liangliang Cao, Jiawei Han, Chengxiang Zhai, and Thomas S. Huang
In: WWW ACM (2011) , p. 247-256.
A Nonparametric Bayesian Model of Multi-Level Category Learning.
by: Kevin Robert Canini, and Thomas L. Griffiths
In: AAAI AAAI Press (2011) .
Naveed, Nasir; Gottron, Thomas; Sizov, Sergej; Staab, Steffen (2012): FREuD: Feature-Centric
Sentiment Diversification of Online Discussions. In: WebSci'12: Proceedings of the 4th International
Conference on Web Science. ACM, 2012.
Nasir Naveed, Sergej Sizov, Steffen Staab: ATTention: Understanding Authors and Topics in Context of
Temporal Evolution. European Conference on Information Retrieval 2011: 733-737. Springer, 2011.
Further papers about our work currently in preparation. Contact us if interested
WeST Steffen Staab Topic Detection - TNG
27 of 25