Presenter: Prof. Jure Leskovec, Pinterest & Stanford

1. Jure Leskovec (@jure)
Pinterest and Stanford
1Jure Leskove, Pinterest & Stanford University

2. 2
Recommendations
drive whole businesses!
Jure Leskove, Pinterest & Stanford University

3. People and Items
3
+
–
+
+
+
–
People Items
Fundamental problem:
Making items discoverable!Jure Leskove, Pinterest & Stanford University

4. Understanding Products
To make relevant recommendations
we need to understand the products
and how they fit together
Discovering relationships
between products
4Jure Leskove, Pinterest & Stanford University

5. Product Graph
Ingest product catalogs:
 10s of millions of products
 100s of millions of descriptions, reviews
Infer product networks with multiple
types of directed relationships:
 Input:
Data about items (products)
 Output:
Network with multiple types of relationships
5Jure Leskove, Pinterest & Stanford University

6. Product Graph: Relations
6
Substitutes:
Purchase
instead
Complements:
Purchase
in addition
Jure Leskove, Pinterest & Stanford University

7. Product Graph: Description
7
: cleaner; quieter
: cheaper; high power
: well made, easy to install
: fits perfectly, great value
Jure Leskove, Pinterest & Stanford University

8. Product Graph: Overview
8
substitute
complement
Jure Leskove, Pinterest & Stanford University

9. Product Graph:What it does?
1. Understands the notions of
substitute and complement goods
is substitutable for
complements
9Jure Leskove, Pinterest & Stanford University

10. Product Graph:What it does?
2. Generates explanations
of why certain products are
preferred
“Good quality, soft, light
weight, the colors are
beautiful and exactly like
the picture!”
People prefer this
because:
10Jure Leskove, Pinterest & Stanford University

11. Product Graph:What it does?
3. Discovers micro-categories
of products
Small clusters of tightly related products:
11Jure Leskove, Pinterest & Stanford University

12. Product Graph:What it does?
4. Recommends
baskets of related products
Query: Suggested outfit:
Query: Suggested outfit:
12Jure Leskove, Pinterest & Stanford University

13. Product Graph: Overview
Building networks from products
Modeling: Can we use product data
to model product relationships?
Understanding: Can we explain
why people prefer certain products
over others?
13Jure Leskove, Pinterest & Stanford University

14. Problem Setting
Binary prediction task:
Given a pair of products, x and y, predict
whether they are related
(substitute/complementary)
Goal: Build a probabilistic model
that encodes
14Jure Leskove, Pinterest & Stanford University

15. Problem Setting
How to learn
from data
Train by maximum likelihood:
15
XComplementary
Not
Complementary
Jure Leskove, Pinterest & Stanford University

16. Approach
Products are described by their properties:
 Review text, Product description,
Brand, Price, …
[0.3, 0, 0, 0.3, 0, 0, 0.2, 0, 0, 0.1]
[0.1, 0, 0, 0, 0.2, 0, 0, 0.1, 0, 0.2]
Challenges:
 How do we discover right features?
 How do we explain relationships?
 How do we identify micro-categories?
16
Shoes Female
Jure Leskove, Pinterest & Stanford University

17. Our Solution: SCEPTRE
Link Prediction Review “topics”
Discover topics that “explain” product relations
17
Learn to discover topics that
explain the product graph
Jure Leskove, Pinterest & Stanford University

18. Challenges: Relation Direction
why do people who view
X eventually buy Y?
Relationships we want to learn
are not symmetric
18
Relationships: Explained by product “properties”
“baby, pajamas, pants, colorful”
Directedness: Subjective/qualitative language
“true size, fits well, items are the same color as on the picture”
Jure Leskove, Pinterest & Stanford University

19. Challenges: Multiple Relations
19
We want to learn multiple
relationships simultaneously
Solution: Learn multiple regressors (one for each
graph), that operate on a single set of topics
Jure Leskove, Pinterest & Stanford University

20. Challenges: Micro-Categories
20
Model discovers thousands of topics
but no micro-categories
Solution: Product hierarchy
Laptop charger specific topics
are only active for chargers.
These are micro-categories.
Topics at the top are common to all
electronics products, and will contain
generic electronics language
Associate each node in the category
tree with a small number of topics:
Jure Leskove, Pinterest & Stanford University

21. Building the Graph
C++ implementation that runs
on a single (large-memory) machine
 OpenMP to parallelize computations
Experimental results:
Active part of the Amazon catalog
 10m products
 150m reviews
 250m relationships
21Jure Leskove, Pinterest & Stanford University

22. Example: Product Graph
22Jure Leskove, Pinterest & Stanford University

23. Example: Product Graph
23Jure Leskove, Pinterest & Stanford University

24. Edge Prediction Accuracy
24
Substitute Complement
Men’s
Clothing
96.7% 94.1%
Women’s
Clothing
95.9% 94.1%
Books 93.8% 89.9%
Electronics 95.7% 88.8%
Movies 85.6% -
Music 90.4% -
OVERALL 94.83% 90.23%
Jure Leskove, Pinterest & Stanford University

25. Results: Micro-Categories
25Jure Leskove, Pinterest & Stanford University

26. 27
How does all this fit
into Pinterest?
Jure Leskove, Pinterest & Stanford University

27. Connecting People & Objects
28Jure Leskove, Pinterest & Stanford University

28. Pins: Richly Annotated Objects
29Jure Leskove, Pinterest & Stanford University

29. Pins are Collected in Boards
30Jure Leskove, Pinterest & Stanford University

30. 30+ Billion Pins
categorized by people into more than
750 Million Boards
50% of pins have been created
in the last 6 months
31

31. 32
Discovering relationships
between objectsJure Leskove, Pinterest & Stanford University

32. We are hiring!
33
jure@pinterest.com

33. References
 Inferring Networks of Substitutable and Complementary
Products by J. McAuley, R. Pandey, J. Leskovec. ACM SIGKDD
International Conference on Knowledge Discovery and Data
Mining (KDD), 2015.
 Hidden Factors and Hidden Topics: Understanding Rating
Dimensions with Review Text by J. McAuley, J. Leskovec. ACM
Conference on Recommender Systems (RecSys), 2013.
 Learning Attitudes and Attributes from Multi-Aspect
Reviews by J. McAuley, J. Leskovec, D. Jurafsky. IEEE
International Conference On Data Mining (ICDM), 2012.
34Jure Leskove, Pinterest & Stanford University