This document provides an overview of classification techniques in machine learning, including:
1. It reviews binary classification problems and how logistic regression can be used to predict the probability that an input belongs to a particular class.
2. It then discusses multi-class classification problems with more than two classes and how logistic regression can be extended to provide probabilities for each class.
3. Examples are given of classifying tumors as malignant or benign and predicting whether a user has a flip phone based on age and income. Visualizations of logistic regression are also presented.
2. Lesson Plan:
1. Homework Survey and Announcements
2. Review
a. Loss Function
b. Gradient Descent
c. Classification vs Regression
3. Binary Classification
4. Multi Class Classification
3. Check in: Fill out the Homework Survey!
Announcements:
- Practical AI Syllabus
11. Loss: A measure of how bad your function is doing
Which Picture Has More Loss?
12. Loss is based on your line’s slope and its y-intercept
Grade
Homework
hours per
week
w = 1/3
b = 5
13. Loss Function
We then build a loss function that takes in w
and b and spits out the loss associated with
the line y = wx + b.
For y = wx + b, loss(w, b).
(y = wx + b)
14. B -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0
W
2.0
1.5
1.0
0.5
0.0
-0.5
-1.0
-1.5
-2.0
(y = wx + b)
29. Logistic regression: want model to predict a
probability of belonging to a class
Probability of being a cat = 0.6
30. Logistic regression is a regression problem
because it predicts a numerical value (i.e. the
probability between 0 and 1) for an input.
If we “round” the probability to the nearest class
(i.e. 0.97 -> 1, and 0.24 -> 0), then we get binary
classification.
31. Classifying a tumor as malignant or benign
Tumor Size
example from Andrew Ng’s lecture 6
Probability
it’s Malignant
32. Let’s have the y-axis be the probability that the
tumor is malignant.
Tumor Size
0
1
Probability
it’s Malignant
33. What happens if we try linear regression?
Tumor Size
Probability
it’s Malignant
0
1
34. What happens if we try linear regression?
Tumor Size
0
1
thresholdProbability
it’s Malignant
36. Using the sigmoid function, we can squash the
output into the range [0,1].
y = mx + b
0
1
(Predicting Malignancy
Based on Tumor Size)
Malignancy0
Probability
it’s Malignant
41. Using the sigmoid function, we can squash the
output into the range [0,1].
y = mx + b
Probability
it’s Malignant
0
1
Malignancy0
(Predicting Malignancy
Based on Tumor Size)
