# Least squares modelling

## Learning outcomes

• Understand the difference between correlation and covariance.
• What the objective function of least squares does
• Understand and use an analysis of variance table
• Calculate and interpret the confidence intervals from a least squares model
• Know about the assumptions required to interpret least squares model coefficients
• Use the prediction error range from the model
• Identify outlier points and classify them
• Use the linear model when there are multiple predictor variables (this is what we are building up towards; we will use multiple linear regression extensively in the next topic)

## Resources

Tasks to do first Quiz Solution
Watch videos 1, 2, and 3, and complete step 16 of the Software tutorial Quiz Solution
Watch videos 4, 5 and 6, and also see the Thermocouple example code (below) Quiz Solution
Watch videos 7 and 8, and complete steps 17 and 18 of the Software tutorial Quiz Solution
Watch videos 9, 10 and 11, and complete step 19 of the Software tutorial Quiz Solution
Watch videos 12, 13, 14 and 15, and complete steps 21 and 22 of the Software tutorial Quiz Solution
• Advanced students should read in the course textbook about leverage, discrepancy, and influential points. Once you've done that, please complete step 20 of the Software tutorial and use the code below to detects points with high leverage, discrepancy, and influence.

• Run the code below to see how to build and use a linear model in R, but see step 16 and onwards in the R tutorial as well.
• Try some practice problems.
• Describe a linear regression model you have made for a lab report.
• What was the $$R^2$$ value?
• How did you calculate the regression model values?
• Use the same data from your report and instead calculate the standard error, $$S_E$$. How do you interpret that $$S_E$$ value now?
• Do the YouTube challenge: find a video on YouTube that explains the central limit theorem, or the confidence interval, or least squares in a way that is different to explained in class (hopefully you find better explanations than mine). Share the link with a friend in your class.
• Does eating chocolate lead to winning a Nobel prize?

## Class videos from prior years

### Videos from 2015

Watch all these videos in this YouTube playlist

1. Overview of this module [02:59]
2. Covariance and correlation [09:46]
3. Why least squares, and some other alternatives [covered in class]
4. Some of the math behind the LS model [09:44]
5. Understanding the analysis of variance (ANOVA) [11:58]
6. Interpretation of standard error [covered in class]
7. Assumptions to derive LS confidence intervals [05:45]
8. Confidence intervals interpreted and used in 3 LS examples [11:54]
9. Prediction intervals from the least squares model [04:24]
10. Checking for violations of the least squares assumptions (1 of 2) [07:27]
11. Checking for violations of the least squares assumptions (2 of 2) [11:46]
12. Introducing multiple linear regression - why we need to use it [2:59]
13. MLR - the matrix equation form and an example [11:25]
14. Interpreting the MLR model coefficients and confidence intervals [04:58]
15. Integer variables in the multiple linear regression model [09:51]
 Covered in class | No video | Script
 Covered in class | No video | Script

## Software codes for this section

### Build a multiple linear regression in R (manually and automatically)

• Compare the manual and automatic results to ensure the model coefficients are the same.
• Also see the example after this one
• Run this code in a web-browser

# Calculate the model manually x1.raw <- c(1,3,4,7,9,9) x2.raw <- c(9,9,6,3,1,2) y.raw <- c(3,5,6,8,7,10) n <- length(x1.raw) x1 <- x1.raw - mean(x1.raw) x2 <- x2.raw - mean(x2.raw) y <- y.raw - mean(y.raw) X <- cbind(x1, x2) # Calculate: b = inv(X'X) X'y XTX <- t(X) %*% X # compare this to cov(X)*(n-1) XTY <- t(X) %*% y XTX.inv <- solve(XTX) b <- XTX.inv %*% XTY b # Let R calculate the model: model <- lm(y.raw ~ x1.raw + x2.raw) summary(model)

### Build the multiple linear regression in R for video 13

x1 <- c(-4.5, -2.5, -1.5, 1.5, 3.5, 3.5) x2 <- c( 4, 4, 1, -2, -4, -3) y <- matrix(c(-3.5, -1.5, -0.5, 1.5, 0.5, 3.5), nrow=6, ncol=1) X <- cbind(x1, x2) XtX <- t(X) %*% X Xty <- t(X) %*% y XtX.inv <- solve(XtX) b <- XtX.inv %*% Xty summary(lm(y ~ x1 + x2))