Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
On Search, Personalisation and 
Real-time Advertising 
Dr. Jun Wang, Senior Lecturer 
Computer Science, University College...
Summary 
30/10/14 
Dunnhumby 
Talk 
2
Web search Ads (display 
opportunities) 
- Maximize 
profit 
Search results 
- Maximize users’ 
satisfactions? 
Query 
30/...
Recommder 
Systems 
30/10/14 
Dunnhumby 
Talk 
4
Recommeder 
Systems 
Kruschwitz, 
Udo 
<udo@essex.ac.uk> 
30/10/14 
Dunnhumby 
Talk 
5
Real-­‐2me 
Adver2sing
Real-­‐2me 
Adver2sing 
7
Real-­‐2me 
Adver2sing 
“This 
is 
Lawrence 
from 
India. 
I 
was 
searching 
Recommender 
model 
in 
web 
and 
found 
you...
Summary 
30/10/14 
Dunnhumby 
Talk 
9
Search 
(Informa2on 
Retrieval) 
l General 
definiJon: 
search 
large-­‐scale 
unstructured 
data, 
mostly 
text 
documen...
Queries 
can 
have 
ambiguous 
intents 
[Courtesy 
of 
F. 
Radlinski, 
MSR 
Cambridge] 
Columbia 
clothing/sportswear 
Col...
Diversified 
search 
results 
Diversifica2on 
-­‐> 
nega2ve 
correla2on 
-­‐> 
reduce 
the 
risk: 
see 
our 
sigir09 
pape...
Recall 
driven 
personalised 
search: 
relevance 
feedback 
revisit 
• www13 
paper 
exploratory 
relevance 
ranking 
Xiao...
Recall 
driven 
search: 
relevance 
feedback 
revisit 
• www13 
paper 
Exploratory 
ranking 
Personalised 
re-­‐ranking 
X...
ranking. We let s represent all rank actions s1 . . . sT. We 
denote r = [r1, . . . , rK] as the vector of feedback inform...
How 
it 
works 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
o 
o 
o 
30/10/14 
Dunnhumby 
Talk 
o 
o 
o 
o 
¤ 
¤ 
¤ 
¤ 
¤ 
¤ 
¤...
How 
it 
works 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
o 
o 
o 
Q 
30/10/14 
Dunnhumby 
Talk 
o 
o 
o 
o 
¤ 
¤ 
¤ 
¤ 
¤ 
¤...
How 
it 
works 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
o 
o 
o 
Q 
30/10/14 
Dunnhumby 
Talk 
o 
o 
o 
o 
¤ 
¤ 
¤ 
¤ 
¤ 
¤...
How 
it 
works 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
o 
o 
o 
Q 
-­‐1 
-­‐1 
30/10/14 
Dunnhumby 
Talk 
o 
o 
o 
o 
¤ 
¤ 
¤...
How 
it 
works 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
o 
o 
o 
Q 
-­‐1 
-­‐1 
30/10/14 
Dunnhumby 
Talk 
o 
o 
o 
o 
¤ 
¤ 
¤...
How 
it 
works 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
x 
o 
o 
o 
Q 
-­‐1 
-­‐1 
30/10/14 
Dunnhumby 
Talk 
o 
o 
o 
o 
¤ 
¤ 
¤...
Summary 
30/10/14 
Dunnhumby 
Talk 
22
Is 
Personalized 
Rec. 
Always 
BeFer? 
Non-­‐personalized 
Top 
Ar2sts 
in 
October 
Personalized: 
Ar2sts 
Recommended 
...
Personalized 
vs 
Non-­‐Personalized 
Dataset: 
Movielens-­‐100k
Personalized 
vs 
Non-­‐Personalized 
• Personalized 
top-­‐N 
CF 
as 
a 
learning 
model 
– Improve 
the 
object 
of 
ove...
Switch 
between 
Two 
PorQolios 
Porqolio 
DominaJon
Case 
Study 
1 
• Demography: 
50 
years 
old 
male 
programmer 
• History: 
86 
feedbacks, 
most 
of 
which 
are 
unpopul...
Case 
Study 
1 
• Analysis: 
BPR 
porqolio 
dominates 
POP 
porqolio 
• Results: 
BPR 
has 
beger 
ranking 
performance
Case 
Study 
2 
• Demography: 
28 
years 
old 
male 
engineer 
• History: 
15 
feedbacks, 
most 
of 
which 
are 
popular 
...
Case 
Study 
2 
• Analysis: 
POP 
porqolio 
dominates 
BPR 
porqolio 
• Results: 
POP 
has 
beger 
ranking 
performance
Cold-­‐start 
problem 
in 
recommmender 
systems
Interac2ve 
Recommender 
Systems
Possible 
Solu2ons 
Zhao, 
Xiaoxue, 
Weinan 
Zhang, 
and 
Jun 
Wang. 
"InteracJve 
collaboraJve 
filtering." 
CIKM, 
2013.
Objec2ve 
Interac2ve 
Cold-­‐start 
problem 
mechanism 
for 
CF 
Zhao, 
Xiaoxue, 
Weinan 
Zhang, 
and 
Jun 
Wang. 
"Intera...
Proposed 
EE 
algorithms 
Thompson 
Sampling 
Linear-­‐UCB 
General 
Linear-­‐UCB 
Zhao, 
Xiaoxue, 
Weinan 
Zhang, 
and 
J...
Cold-­‐start 
users 
Zhao, 
Xiaoxue, 
Weinan 
Zhang, 
and 
Jun 
Wang. 
"InteracJve 
collaboraJve 
filtering." 
CIKM, 
2013...
Summary 
30/10/14 
Dunnhumby 
Talk 
37
Real-­‐2me 
Adver2sing
Life 
of 
a 
display 
ad 
in 
the 
RTB 
environment: 
0.36 
seconds 
39 
Ad 
Exchange 
Demand-Side 
Platform 
Advertiser 
...
DSP 
(Demand 
Side 
PlaQorm) 
30/10/14
Bidder 
in 
DSP
Op2mal 
Bidder: 
Problem 
Defini2on 
Bid 
Request 
Bid 
Engine 
Bid 
Price 
42 
Input: 
bid 
request 
include 
Cookie 
inf...
43 
The 
General 
Process 
for 
Bidding 
Op2misa2on 
Red: 
hard 
constraints 
Green: 
features 
Blue: 
models 
Note 
that ...
Op2mal 
bidder: 
the 
formula2on 
• FuncJonal 
OpJmisaJon 
Problem 
– Dependency 
assumpJon: 
• SoluJon: 
Calculus 
of 
va...
Op2mal 
bidder: 
the 
solu2on 
Weinan 
Zhang, 
Shuai 
Yuan, 
Jun 
Wang, 
OpJmal 
Real-­‐Time 
Bidding 
for 
Display 
Adver...
Experiments 
Offline 
Online 
Winner 
of 
the 
first 
global 
Real-­‐Jme 
Bidding 
algorithm 
contest 
2013-­‐2014 
Weinan...
UCL 
OpenBidder 
Benchmarking 
System 
47
time (see Figure 1). This makes the cost of displaying ad slots (for 
advertisers) and the advertising incomes (for publis...
Automa2ng 
Ads 
Futures/Op2on 
Contracts 
• Need 
Ad’s 
Futures 
Contract 
and 
Risk-­‐reduc5on 
Capabili5es 
– Technologi...
Futures 
Exchange 
(Programma2c 
Guarantee) 
Advertiser 
Demand 
Side 
Platform 
(DSP) 
Futures 
Exchange 
RTB / Spot 
Exc...
Acknowledgements 
• Thanks 
to 
my 
PhD 
students 
Weinan 
Zhang, 
Shuai 
Yuan, 
Marc 
Sloan, 
Xiaoxue 
Zhao 
30/10/14 
Du...
For 
more 
informa2on, 
please 
refer 
to 
1. Wang, 
Jun, 
and 
Jianhan 
Zhu. 
"Porqolio 
theory 
of 
informaJon 
retrieva...
Thanks 
for 
your 
aFen2on 
53
Prochain SlideShare
Chargement dans…5
×

On Search, Personalisation and Real-time Advertising

The talk presented in Dunnhumby.

  • Soyez le premier à commenter

On Search, Personalisation and Real-time Advertising

  1. 1. On Search, Personalisation and Real-time Advertising Dr. Jun Wang, Senior Lecturer Computer Science, University College London Email: j.wang@cs.ucl.ac.uk Twitter: @seawan 30/10/14 Dunnhumby Talk 1
  2. 2. Summary 30/10/14 Dunnhumby Talk 2
  3. 3. Web search Ads (display opportunities) - Maximize profit Search results - Maximize users’ satisfactions? Query 30/10/14 Dunnhumby Talk 3
  4. 4. Recommder Systems 30/10/14 Dunnhumby Talk 4
  5. 5. Recommeder Systems Kruschwitz, Udo <udo@essex.ac.uk> 30/10/14 Dunnhumby Talk 5
  6. 6. Real-­‐2me Adver2sing
  7. 7. Real-­‐2me Adver2sing 7
  8. 8. Real-­‐2me Adver2sing “This is Lawrence from India. I was searching Recommender model in web and found your webpage in search engine. Then, I visited your webpage searching relevant contents and saw unrelevant Google add in "Research Team" page (aFached screenshot). This add might vary from country to country. But I feel it will mislead and give wrong opinion to users who visit your webpage.” -­‐ Lawrence from India
  9. 9. Summary 30/10/14 Dunnhumby Talk 9
  10. 10. Search (Informa2on Retrieval) l General definiJon: search large-­‐scale unstructured data, mostly text documents, but also include images, videos, etc l ApplicaJons: – web search – product search – enterprise search – desktop/email search – informaJon filtering – collaboraJve filtering and recommeder systems 30/10/14 Dunnhumby Talk 10
  11. 11. Queries can have ambiguous intents [Courtesy of F. Radlinski, MSR Cambridge] Columbia clothing/sportswear Colombia (Country: misspelling) Columbia University Columbia Records music/video columbia 30/10/14 Dunnhumby Talk 11
  12. 12. Diversified search results Diversifica2on -­‐> nega2ve correla2on -­‐> reduce the risk: see our sigir09 paper 30/10/on 14 porQolio theory of informaDunnhumby 2on Talk retrieval 12
  13. 13. Recall driven personalised search: relevance feedback revisit • www13 paper exploratory relevance ranking Xiaoran Jin, Marc Sloan, and Jun Wang. InteracJve Exploratory Search for MulJ Page Search Results, www13 Figure 1: Example application, where Page 1 contains the Page 2 contains a refined, personalised re-ranking of the Personalised re-­‐ranking 30/10/14 Dunnhumby Talk 13
  14. 14. Recall driven search: relevance feedback revisit • www13 paper Exploratory ranking Personalised re-­‐ranking Xiaoran Jin, Marc Sloan, and Jun Wang. InteracJve Exploratory Search for MulJ Page Search Results, www13 30/10/14 Dunnhumby Talk 14 contains the diversified, exploratory relevance ranking, and
  15. 15. ranking. We let s represent all rank actions s1 . . . sT. We denote r = [r1, . . . , rK] as the vector of feedback informa-tion Recall obtained driven from the user search: for a given page, relevance where K is the number of documents given feedback ri is the feedback feedback information gained revisit with 0  K  M, and (the rating provided by the user) of relevance feedback for document i, either by measuring a direct rating or by observing clickthroughs. We use a weighted sum of the expected DCG@M scores of the rankings of the T upcoming result pages, denoted here by (note that Rst • We consider MulJ Page Search Results • Intend to opJmise overall expected effecJveness over the search journey • Our j ⌘ Rt st j ) Us = XT t 0 @!t XtM j=1+(t−1)M derivaJon shows that to represent the user’s overall satisfaction, where E(Rst – Page E(Rst j ) log2(j + 1) 1 A (2) ) = 1 contains the diversified, exploratory relevance ✓st is the expected relevance of a document at rank j in ranking – Page j result page t. We have chosen the objective function as it is simple and both rewards finding the most relevant docu-ments 2 contains, personalised re-­‐ranking of the next j and also ranking them in the correct order, although set of remaining documents, where the relevance feedback other IR metrics is triggered can be adopted by the similarly. “Next” The burank gon weight 1 log2 j is used to give greater weight to ranking the most rele-vant documents in higher positions. The tunable parameter !i # 0 is used to adjust the importance of result pages and thus the level of exploration in the initial page(s). When !1 U1 Figure by the diagram, random the rank node is conditional the feedback P(R2= where rsat Xiaoran Jin, Marc Sloan, and Jun Wang. InteracJve Exploratory Search for MulJ Page Search Results, www13 30/10/14 Dunnhumby Talk 15
  16. 16. How it works x x x x x x x x x x x o o o 30/10/14 Dunnhumby Talk o o o o ¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤ x x X: doc about apple fruit doc about apple ceo ¤ ¤ O: doc about apple iphone Page 1: diversified, exploratory relevance ranking considers Relevancy + Variance + |CorrelaJons| Page 2: personalised re-­‐ranking 16
  17. 17. How it works x x x x x x x x x x x o o o Q 30/10/14 Dunnhumby Talk o o o o ¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤ x x X: doc about apple fruit doc about apple ceo ¤ ¤ O: doc about apple iphone Page 1: diversified, exploratory relevance ranking considers Relevancy + Variance + |CorrelaJons| 17
  18. 18. How it works x x x x x x x x x x x o o o Q 30/10/14 Dunnhumby Talk o o o o ¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤ x x X: doc about apple fruit doc about apple ceo ¤ ¤ O: doc about apple iphone Page 1: diversified, exploratory relevance ranking considers Relevancy + Variance + |CorrelaJons| 18
  19. 19. How it works x x x x x x x x x x x o o o Q -­‐1 -­‐1 30/10/14 Dunnhumby Talk o o o o ¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤ x x X: doc about apple fruit doc about apple ceo ¤ ¤ O: doc about apple iphone +1 Page 1: diversified, exploratory relevance ranking considers Relevancy + Variance + |CorrelaJons| 19
  20. 20. How it works x x x x x x x x x x x o o o Q -­‐1 -­‐1 30/10/14 Dunnhumby Talk o o o o ¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤ x x X: doc about apple fruit doc about apple ceo ¤ ¤ O: doc about apple iphone Page 2: Personalised reranking: +1 Q 20
  21. 21. How it works x x x x x x x x x x x o o o Q -­‐1 -­‐1 30/10/14 Dunnhumby Talk o o o o ¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤ x x X: doc about apple fruit doc about apple ceo ¤ ¤ O: doc about apple iphone Page 2: Personalised reranking: +1 Q 21
  22. 22. Summary 30/10/14 Dunnhumby Talk 22
  23. 23. Is Personalized Rec. Always BeFer? Non-­‐personalized Top Ar2sts in October Personalized: Ar2sts Recommended for You
  24. 24. Personalized vs Non-­‐Personalized Dataset: Movielens-­‐100k
  25. 25. Personalized vs Non-­‐Personalized • Personalized top-­‐N CF as a learning model – Improve the object of overall relevance – But does NOT improve on each user POP BPR Top-­‐N Performance Low bias High variance
  26. 26. Switch between Two PorQolios Porqolio DominaJon
  27. 27. Case Study 1 • Demography: 50 years old male programmer • History: 86 feedbacks, most of which are unpopular items
  28. 28. Case Study 1 • Analysis: BPR porqolio dominates POP porqolio • Results: BPR has beger ranking performance
  29. 29. Case Study 2 • Demography: 28 years old male engineer • History: 15 feedbacks, most of which are popular items
  30. 30. Case Study 2 • Analysis: POP porqolio dominates BPR porqolio • Results: POP has beger ranking performance
  31. 31. Cold-­‐start problem in recommmender systems
  32. 32. Interac2ve Recommender Systems
  33. 33. Possible Solu2ons Zhao, Xiaoxue, Weinan Zhang, and Jun Wang. "InteracJve collaboraJve filtering." CIKM, 2013.
  34. 34. Objec2ve Interac2ve Cold-­‐start problem mechanism for CF Zhao, Xiaoxue, Weinan Zhang, and Jun Wang. "InteracJve collaboraJve filtering." CIKM, 2013.
  35. 35. Proposed EE algorithms Thompson Sampling Linear-­‐UCB General Linear-­‐UCB Zhao, Xiaoxue, Weinan Zhang, and Jun Wang. "InteracJve collaboraJve filtering." CIKM, 2013.
  36. 36. Cold-­‐start users Zhao, Xiaoxue, Weinan Zhang, and Jun Wang. "InteracJve collaboraJve filtering." CIKM, 2013.
  37. 37. Summary 30/10/14 Dunnhumby Talk 37
  38. 38. Real-­‐2me Adver2sing
  39. 39. Life of a display ad in the RTB environment: 0.36 seconds 39 Ad Exchange Demand-Side Platform Advertiser Data Management Platform 0. Ad Request 1. Bid Request (user, context) 2. Bid Response (ad, bid) 4. Win NoJce 3. Ad AucJon (paying price) 5. Ad (with tracking) 6. User Feedback (click, conversion, etc.) User InformaJon User Demography: Male, 25, Student, etc. User SegmentaJons: Ad science, London, etc. Webpage User
  40. 40. DSP (Demand Side PlaQorm) 30/10/14
  41. 41. Bidder in DSP
  42. 42. Op2mal Bidder: Problem Defini2on Bid Request Bid Engine Bid Price 42 Input: bid request include Cookie informaJon (anonymous profile), website category & page, user terminal, locaJon etc Output: bid price Considera2ons: Historic data, CRM (first party data), DMP (3rd party data from Data Management Plaqorm) What is the op2mal bidder given a budget constraint? e.g., Maximise Subject to the budget constraint
  43. 43. 43 The General Process for Bidding Op2misa2on Red: hard constraints Green: features Blue: models Note that “Frequency & recency rules” are also used as features
  44. 44. Op2mal bidder: the formula2on • FuncJonal OpJmisaJon Problem – Dependency assumpJon: • SoluJon: Calculus of variaJons context+ad features winning funcJon CTR esJmaJon bidding funcJon Weinan Zhang, Shuai Yuan, Jun Wang, OpJmal Real-­‐Time Bidding for Display AdverJsing, KDD’14
  45. 45. Op2mal bidder: the solu2on Weinan Zhang, Shuai Yuan, Jun Wang, OpJmal Real-­‐Time Bidding for Display AdverJsing, KDD’14
  46. 46. Experiments Offline Online Winner of the first global Real-­‐Jme Bidding algorithm contest 2013-­‐2014 Weinan Zhang, Shuai Yuan, Jun Wang, OpJmal Real-­‐Time Bidding for Display AdverJsing, KDD’14
  47. 47. UCL OpenBidder Benchmarking System 47
  48. 48. time (see Figure 1). This makes the cost of displaying ad slots (for advertisers) and the advertising incomes (for publishers and search engines) unpredictable. (RTB) Ads Thus prices there are increasing are volaneeds 2of le a new advertising trading mechanism to manage the risk of cost or income. (a) The price movement of a display opportunity from Yahoo! ads data Under GSP (generalized second price aucJon) 50 Ad slot price (GSP) 1.5 1 Price change rate 30/10/14 Dunnhumby Talk 48
  49. 49. Automa2ng Ads Futures/Op2on Contracts • Need Ad’s Futures Contract and Risk-­‐reduc5on Capabili5es – Technologies are constrained mainly to “spots” markets, i.e., any transacJon where delivery takes place right away (in Real-­‐Jme AdverJsing and Sponsored Search) – No principled technologies to support efficient forward pricing &risk management mechanisms • If we got Futures Market or provide Op2on Contracts, adverJsers could lock in the campaign cost and Publishers could lock in a profit in the future 30/10/14 Dunnhumby Talk 49
  50. 50. Futures Exchange (Programma2c Guarantee) Advertiser Demand Side Platform (DSP) Futures Exchange RTB / Spot Exchange Supply Side Platform (SSP) Publisher 3rd party data providers, ad serving, ad agency, ad networks, campaign analytics -10% to -30%
  51. 51. Acknowledgements • Thanks to my PhD students Weinan Zhang, Shuai Yuan, Marc Sloan, Xiaoxue Zhao 30/10/14 Dunnhumby Talk 51
  52. 52. For more informa2on, please refer to 1. Wang, Jun, and Jianhan Zhu. "Porqolio theory of informaJon retrieval." SIGIR, 2009. 2. Jin, Xiaoran, Marc Sloan, and Jun Wang. "InteracJve exploratory search for mulJ page search results." WWW, 2013. 3. Zhang, Weinan, et al. "To personalize or not: a risk management perspecJve." Proceedings of the 7th ACM conference on Recommender systems. ACM, 2013. 4. Gorla, Jagadeesh, et al. "ProbabilisJc group recommendaJon via informaJon matching." WWW, 2013. 5. Shuai Yuan, Jun Wang, Real-­‐Jme Bidding for Online AdverJsing: Measurement and Analysis, AdKDD’13 hgp://arxiv-­‐web3.library.cornell.edu/abs/1306.6542 6. Weinan Zhang, Shuai Yuan, Jun Wang, OpJmal Real-­‐Time Bidding for Display AdverJsing, KDD’14 7. Shuai Yuan, Jun Wang, Bowei Chen, An Empirical Study of Reserve Price OpJmisaJon in Real-­‐Time Bidding 8. Bowei Chen, Jun Wang, Ingemar Cox, and Mohan Kankanhalli, MulJ-­‐ Keyword MulJ-­‐Click OpJon Contracts for Sponsored Search AdverJsing, under submission, 2013 hgp://arxiv.org/abs/1307.4980
  53. 53. Thanks for your aFen2on 53

×