Natural Language Understanding @ Facebook Scale: At Facebook, text understanding is the key to surfacing content that’s relevant and personalized, plus enabling new experiences like social recommendations and Marketplace suggestions. In this talk, I will introduce you to DeepText, Facebook’s platform for text understanding, and discuss the various models it supports. Bio: Rushin Shah is an engineering leader at Facebook, currently working on natural language understanding and dialog. Previously, he was at Siri at Apple for 5 years, where he built and headed the natural language understanding group, which included teams dedicated to modeling, engineering and data science. He also worked at the query understanding group at Yahoo. He has worked on a broad range of problems in the NLP area including parsing, information extraction, dialog and question answering. He holds degrees in language technologies and computer science from Carnegie Mellon and IIT Kharagpur.

1. Natural Language Understanding
@ Facebook Scale
ENGINEERING MANAGER, FB
Rushin Shah

2. Our Goal:
Understand textual content with near
human accuracy at Facebook scale

3. Friends ML Conferences
¬
Hockey
The Daily Puck
The Canadiens ran through a high-intensity practice on
Wednesday ahead of Thursday's game.
1 hr •
Ron Timpany, Dan Fell and 80 others
Like Comment Share
Montreal Hockey Insider
Canadiens on verge of clinching playoff spot, Lindgren
called up.
1 hr •
Bill Russell, Joe Tony and 116 others
Like Comment Share
Benoit Dumoulin was watching Vancouver
Canucks vs. Montreal Canadiens.
Yes! In OT!
1 hr •
Bill Russell, Joe Tony and 234 others
Like Comment Share
VAN 3 FINAL OT 4 MTL

4. NLU Tasks
• Text classification
• Word classification
• Content similarity
• Entity Resolution

5. Text Classification
Entity: Delicious Food
Jole Simmons
1 hr •
Sarah Russell and 23 others 4 Comments
Like Comment Share
Topic: Cooking
I'm trying out this new recipe for a coconut curry
tonight. It looks DELICIOUS!!!
Entity: Kaepernick
Jole Simmons
1 hr •
Sarah Russell and 23 others 4 Comments
Like Comment Share
Topic: Sports or Cooking ?
Last night's game was absolutely incredible. Once
Curry gets cooking, there's no stopping that guy!

6. Word Classification

7. Content similarity
SIMILARITY
0.75

8. MUCKBUCKET SUNSHINE
MUSIC BAND
Parsing out Entities
"DID YOU KNOW THAT MUCK BUCKET
SUNSHINE IS PERFORMING LIVE AT
THE BOOM BOOM ROOM?"
TOPIC
LIVE EVENT
The Boom Boom Room
1 hr •
Tonight before you Dance Dance Dance, join us for a
special evening. Boom Boom Room Presents:
Muckbucket Sunshine! Doors 8pm.
Sarah Russell and 23 others 4 Comments
Like Comment Share

9. • Deep Learning
For NLU
• Continuous
Representation
• Can Solve Hard
NLP Problems
Natural Language Processing
(Almost) From Scratch

10. DeepText

11. DeepText Features
Multiple Tasks Multiple Languages Multiple Architectures

12. Model Structure Learning Algorithm
Data Loader Tokenizer Feature Extraction
Model Training
DeploymentFeat Prep
DeepText Platform architecture

13. DeepText Tasks
Sequence LabelingClassification

14. DeepText Document classification

15. DeepText Document classification - LSTM
Hidden
MLP
Messi todayscored
HiddenHidden
EmbeddingEmbeddingEmbedding

16. DeepText Document classification - CNN
Messi todayscored
EmbeddingEmbeddingEmbedding
MLP
CNN

17. DeepText Classification: FastText
Messi todayscored
Unigram Bigram Unigram Bigram Unigram
Output
Hidden

18. DeepText Word classification

19. MLP
CNN/LSTM
Messi
Embedding
today
Embedding
Classification(Messi)
MLP
Classification(today)
scored a hat-trick
DeepText Classification: Word classification

20. DeepText Content Similarity
Messi todayscored
EmbeddingEmbeddingEmbedding
CNN / LSTM
Messi todayscored
EmbeddingEmbeddingEmbedding
CNN / LSTM
MLP
Ranking loss

21. FC Barcelona
DeepText Entity recognition and linking
Real Madrid C.F.

22. Document
Candidate
Selection
Mention
Detection
Disambiguation
Entity
Annotations
Entity recognition and linking Architecture

23. Exploring Use Cases In Facebook

24. For Sale
Post for SaleNot Selling
Create a Sales post to sell your items faster. Only post as a discussion if you’re not
selling something.
Are You Selling Something?

25. Social Recs

26. Scale ML
Experts
Reuse
Models
Optimize
Labels

27. CLUE
Label Efficiency
Active Learning
Democratize
Single entry point for NLU
Flexibility and Scale
Deep Text
Scale ML
Experts
Reuse
Models
Optimize
Labels

28. Active
Learning
Self Service With CLUE
Train
Classifier
Label
Data
Collect
Data
(Search)
Review
Threshold
Precision
POOLING
...
W1 W2 W3 W4 WN
CONVOLUTIO
I want some subway or burger king
I want some subway or burger king
Slots
[
{
"name": "cu:restaurant"
"value": "subway"
"start": 12
"end": 19
"contextldx": 0
}
{
"name": "cu:restaurant"
"value": "burger king"
"start": 22
"end": 32
"contextldx": 0

29. MODELS
200+