Today’s information retrieval applications have become increasingly complex. The Social Book Search (SBS) lab at CLEF 2015 allows evaluating retrieval methods on a complex search task with several textual and non-textual meta-data fields. The challenge is to incorporate the different information types (modalities) into a single ranked list. We build a strong textual baseline and combine it with a document prior based on social signals. Further, we include non-textual modalities in relation to the user preferences using random forest learning to rank. Our experiments show that both the social document prior and the learning to rank approach improve the search results.

1. Zürcher Fachhochschule
Melanie Imhof, Ismail Badache, Mohand Boughanem
Université de Neuchâtel, Neuchâtel, Switzerland
Zurich University of Applied Sciences, Winterthur, Switzerland
IRIT - Paul Sabatier University, Toulouse, France

2. Zürcher Fachhochschule
Motivation
• Only the first few “recommendations” are considered
• Many modalities
• Goals
– Fuse textual baseline with non-textual and social
modalities
• Ratings, number of tags, book price and number of pages
– Include user preferences
2

3. Zürcher Fachhochschule
Retrieval Models
3
Learning
to Rank
Textual
Models
Social
Signals-
Based
Model

4. Zürcher Fachhochschule
Textual Models
• Single text field that contains all textual fields
• Query expansion
– Blind relevance feedback (RF)
– Example books with positive and neutral sentiment
• Filter books already read by the topic creator (user
catalog & examples)
4

5. Zürcher Fachhochschule
Social Signals-Based Model
• Social prior probability
• 𝑃 𝐷 = 𝑎 𝑖 ∈𝐴
log 1+ 𝐷 𝑎 𝑖
+ 𝜇 ∙𝑃 𝑎𝑖 𝐶
log 1+ 𝐷 𝑎 + 𝜇
• 𝐷 𝑎 𝑖
is the number of actions of type 𝑎𝑖, e.g. number of tags.
• 𝐷 𝑎 is the number of all actions on document.
• 𝑃 𝑎𝑖 𝐶 probability of 𝑎𝑖 in the collection
• 𝜇 smoothing parameter 5
More popular  higher probability to be relevant
Assumption

6. Zürcher Fachhochschule
Social Signals-Based Model
• Social prior probability for ratings
• 𝑃𝐵𝐴 𝐷 =
1+log(1+𝐵𝐴 𝐷 )
1+log(1+ 𝐷′∈ 𝐶
𝐵𝐴(𝐷′))
• 𝐵𝐴 𝐷 =
𝑎𝑣𝑔 𝐷 𝑟 + 𝐷 𝑟 + 𝐷′∈ 𝐶
𝑎𝑣𝑔 𝐷 𝑟
′ ∙|𝐷 𝑟
′|
𝐷 𝑟 + 𝐷′∈ 𝐶
|𝐷 𝑟
′|
6
More and higher ratings  higher probability to be relevant
Assumption

7. Zürcher Fachhochschule
Learning to Rank (Random Forests)
• Learn how to combine textual and non-textual
modalities into a single ranked list
– Price
– Number of pages
– Ratings
• User preference
– Estimated by the average values in the user’s catalog
– Use difference of document value to user preference
7
~ 190 pages
~30 €

8. Zürcher Fachhochschule
Experimental Evaluation: Runs I
• Textual Model
– Run1: Textual baseline using BM25 with example based
relevance feedback using 35 terms and read book filtering.
• Social Signal-Based Models
– Run3: Run1 using language model combined with
Bayesian average re-ranking based on ratings.
– Run4: Run1 using language model combined with re-
ranking based on the tags.
– Run5: Run1 combined with re-ranking based on the tags
and Bayesian average of ratings. 8

9. Zürcher Fachhochschule
Experimental Evaluation: Runs II
• Random Forests
– Run2: Random forests trained with 10 trees based
on six textual runs and three non-textual
modalities
– Run6: Random forests trained with 100 trees
based on six textual runs and three non-textual
modalities combined with re-ranking based on the
tags and Bayesian average of ratings
9

10. Zürcher Fachhochschule
Results
• Training exceeds no training
• Non-textual modalities contain relevant information
• Examples RF and filtering improve textual baseline
• Social signal prior improves textual baseline
10

11. Zürcher Fachhochschule
Conclusion
• Superiority of social approach compared to textual
model (baseline).
• Test learning approach with completely separated
training and test datasets.
• Find methods that do not rely on learning (cold start
problem).
11