Slides from a lightning talk on classification methods, originally given at Open Source Open Mic Chicago 01/2016. Yes, I know I left things you. You try covering this in 5 minutes.

1. classification edition
Machine Learning in 5 Minutes
Brian Lange

2. hi, i’m a
data scientist

3. classification
algorithms

4. popular examples
-spam filters
-the Sorting Hat

5. things to know
- you need data labeled with the correct answers to
“train” these algorithms before they work
- feature = dimension = attribute of the data
- class = category = Harry Potter house

6. linear discriminants
“draw a line through it”

7. linear discriminants
“draw a line through it”

8. linear discriminants
“draw a line through it”

9. linear discriminants
“draw a line through it”
🎉

10. define what “shitty” means
6 wrong

11. define what “shitty” means
4 wrong

12. a map of shittiness
to find the least shitty line
shittiness
slope
intercept

13. probably don’t use these
linear discriminants:

14. logistic regression
“divide it with a log function”

15. logistic regression
“divide it with a log function”
🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉
+ gives you probabilities
+ the model is a formula
+ can “threshold” to make model more or less
conservative
💩💩💩💩💩💩💩💩💩💩💩
- only works with linear decision boundaries

16. SVMs (support vector machines)
“*advanced* draw a line through it”
- better definition of “shitty”
- lines can turn into non-linear
shapes if you transform your
data

19. 💩

21. 💩

22. “the kernel trick”

24. 🎉
woooooooooooo
🎉🎉

25. SVMs (support vector machines)
“*advanced* draw a line through it”

26. SVMs (support vector machines)
“*advanced* draw a line through it”
🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉
works well on a lot of different shapes of data
thanks to the kernel trick
💩💩💩💩💩💩💩💩💩💩💩
not super easy to explain to people
can only kinda do probabilities

27. KNN (k-nearest neighbors)
“what do similar cases look like?”

28. KNN (k-nearest neighbors)
“what do similar cases look like?”
k=1

29. KNN (k-nearest neighbors)
“what do similar cases look like?”
k=2

30. KNN (k-nearest neighbors)
“what do similar cases look like?”
k=1

31. KNN (k-nearest neighbors)
“what do similar cases look like?”
k=2

32. KNN (k-nearest neighbors)
“what do similar cases look like?”
k=3

33. KNN (k-nearest neighbors)
“what do similar cases look like?”

34. KNN (k-nearest neighbors)
“what do similar cases look like?”
🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉
+ no training, adding new data is easy
+ you get to define “distance”
💩💩💩💩💩💩💩💩💩💩💩
- can be outlier-sensitive
- you have to define “distance”

35. decision tree learners
make a flow chart of it

36. decision tree learners
make a flow chart of it
x < 3?
yes no
3

37. decision tree learners
make a flow chart of it
x < 3?
yes no
y < 4?
yes no
3
4

38. decision tree learners
make a flow chart of it
x < 3?
yes no
y < 4?
yes no
x < 5?
yes no
3 5
4

39. decision tree learners
make a flow chart of it
🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉
+ fit all kinds of arbitrary shapes
+ output is a clear set of
conditionals
💩💩💩💩💩💩💩💩💩💩💩
- extremely prone to overfitting
- have to rebuild when you get new
data
- no probability estimates

40. ensemble models
make a bunch of models and combine them

41. ensemble models
make a bunch of models and combine them
🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉
- don’t overfit as much as their component parts
- Generally don’t require much parameter tweaking
- If data doesn’t change very often, you can make
them semi-online by just adding new trees
- Can provide probabilities
💩💩💩💩💩💩💩💩💩💩💩
- Slower than their component parts (though if
those are fast, it doesn’t matter)