NLP / Chatbot

1. Introduction to NLP
Gil Katz
gil@recast.ai
@gil_katz

2. Introduction to NLP
Gil Katz
gil@recast.ai
@gil_katz

3. Collaborative end-to-end bot platform
BUILD CONNECT MONITORTRAIN

4. 1950: Turing Test (“Computing Machinery and Intelligence” /
Alan Turing)
Historic Background
50’s-60’s: Machine translation to be reality in three to five
years (Georgetown experiment)
60’s-80’s: NLP systems based on hard rules, some quite
impressive!

5. 80’s-90’s: Machine learning for NLP, starting with simple
algorithms (decision trees)
Historic Background
90’s-2000’s: Increasingly statistical models are used (starting
with HMM for POS tagging)
2000’s-today: Neural Networks added to the mix

6. NLU over translation
Written over spoken
Chatbots and Recast.AI
This Talk

7. State of the Art
Intents Entities Skills

8. Intents Subject of communication
Entities Important objects
Skills Take action

9. greetings
I have no internet 4G doesn’t work No signal at my place
report-issue weather

10. report-issuegreetings
I have no internet 4G doesn’t work No signal at my place
weather

11. report-issuegreetings
I have no internet 4G doesn’t work No signal at my place
weather

12. report-issuegreetings
I have no internet 4G doesn’t work No signal at my place
weather
- Where are you located?

13. Powered by Machine Learning

14. Code
ML
data answers
rules
answers
data
rulesdata answers

15. Supervised Unsupervised

16. Classification Regression

17. Preprocessing
Be Prepared!

18. J’ai envie d’une pizza aujourd’hui
J’ai envie d’une pizza aujourd’hui (fr)
J ’ ai envie d ’ une pizza aujourd’hui (fr)
PRON
VERB
NOUN
ADP
DET
ADV
J ’ ai envie d ’ une pizza aujourd’hui (fr)

19. Intent Classification

20. Naive Bayes
Bayes’ Rule: P(i|w) = P(w|i)P(i)/P(w)
‘Sparse’ approach
L’enfant a mangé un poisson
Un poisson a mangé l’enfant
No context management

21. Neural Networks
Words are represented densely in an n-dimensional space
Representations are learned through context
Main problem: How to represent sentences?
vct(Berlin) - vct(Germany) + vct(France) = vct(Paris)

22. SVM
RNNs (LSTM/GRU/QRNN/...)
Bi-LSTM with Attention
What’s more?
Siamese Networks

23. Named Entity Recognition

24. Named Entity Recognition
Sequence Labeling

25. I have no in yesterdaysinceinternet Nancy

26. I have no in yesterdaysinceinternet Nancy
O O O CONNECTION O LOCATION O DATETIME

27. ? ? ? ? ? ? ? ?
I have no in yesterdaysinceinternet Nancy

28. ? ? ? ? ? ? ? ?
I have no in yesterdaysinceinternet Nancy
PERSON
LOCATION

29. ? ? ? ? ? ? ? ?
I have no in yesterdaysinceinternet Nancy
LOCATION

30. Hidden Markov Chains

31. SSVM
CRF
RNN (LSTM/GRU/QRNN/...)
What’s more?

32. Entity Enrichment

33. I have no internet in Nancy since yesterday
{
canonical: …
value: …
}
{
formatted: …
lat: …
lng: …
}
{
formatted: …
iso: …
accuracy: …
}

34. Dialog Management

35. Markov Decision ProcessFinite State Architecture Frame Based

36. User Initiative
Greetings
System Initiative
Payment
Mixed Initiative
QA/Track/Order

37. Wrap Up

38. ML and NLP as engine behind chatbots
A chatbot is built on many ML tasks
Different algorithms fit different tasks
Recast.AI as an end-to-end chatbot development platform

39. Questions?