2. Recap from the last meetup
• statistical learning vs. machine learning
• supervised vs. unsupervised learning
• categorical models vs. quantitative models
7. The line that minimizes the residual
error, or point distance from the line
which is why we refer to the regression line
as the least squares error regression line
8. We determine this with a linear
regression to determine the y-intercept
and the slope of the line that minimizes
the error residuals
Use the lm function in R to create linear models
A regression on one variable is known as a simple
linear regression
10. • relationship of predictors to predicted variables
is linear
• the variance of error terms is constant
(homoskedastic)
• minimal to no outliers in the data (high or low y
in response to x)
• minimal to no leverage points in the data (high
or low x relative to the data)
• no collinearity among predictor variables
• predictors are additive to reliability of model
(no interaction effects)
12. Data Analysis
Bonds dataset from
“A Modern Approach to Regression with R” Sheather, Simon. 2009.
https://link.springer.com/book/10.1007%2F978-0-387-09608-7
Consider the following dataset of bond prices
bonds.dat = read.csv("http://www.stat.tamu.edu/~sheather/book/docs/datasets/bonds.txt", sep='t')
13. Data Analysis
Data analysis is the art of asking
questions of the data and searching
for answers.
What questions should we ask?
• Is Bid Price a function of Coupon
Rate or vice versa?
• What type of relationship does Bid
Price appear to have with Coupon
Rate?
• Is there a formula we could use to
predict Bid Price based on Coupon
Rate?
14. Linear Model of Bond Prices
as a function of Coupon Rates
• Is this line a good or
bad fit with the data?
• Why or why not?
• Can we improve the
model?
• How?
15. Know your data and know
how your models work!
• Why would outliers skew
the linear regression
model?
• What should we do
about it?
• Why do these outliers
exist?
Outliers
16. What are the outliers?
A Flower bond is a U.S treasury bond recoverable
before maturity upon payment or fulfilling a condition, if
used to settle federal estate taxes. When flower bonds
are surrendered in payment of taxes, and accepted as
such, that constitutes payment of those taxes for statute
of limitations and statutory interest purposes.
17. Adjusted Bond Model
• After removing the
outliers the model looks
much better
• But how do you know
that it’s a better model?
20. Residual Sum of Squares (RSS)
“In statistics, the residual sum of squares (RSS), also known as the
sum of squared residuals (SSR) or the sum of squared errors of
prediction (SSE), is the sum of the squares of residuals
(deviations predicted from actual empirical values of data).”
Source: https://en.wikipedia.org/wiki/Residual_sum_of_squares
35. What is the best model?
The most accurate model on the training data is
always the model with the most predictor
variables (p) and the lowest residual sum of
squares (RSS)…but what is the best model?
36. The best model is…
The best model is the simplest model
with the most predictive power on the
entire data population while staying
within your resource constraints.
37. Bonus: Easy and tidy with
the broom package
From the broom vignette:
https://cran.r-project.org/web/packages/broom/vignettes/broom.html
The broom package takes the messy output of built-in functions in R, such as lm,
nls, or t.test, and turns them into tidy data frames.
This package provides three S3 methods that do three distinct kinds of tidying.
• tidy: constructs a data frame that summarizes the model's statistical findings.
This includes coefficients and p-values for each term in a regression, per-cluster
information in clustering applications, or per-test information for multtest
functions.
• augment: add columns to the original data that was modeled. This includes
predictions, residuals, and cluster assignments.
• glance: construct a concise one-row summary of the model. This typically
contains values such as R
2,
adjusted R
2,
and residual standard error that are
computed once for the entire model.
