4. Least Squares Modelling Review

• 4.1. Least squares modelling in context
  • 4.1.1. Usage examples
  • 4.1.2. What you will be able to do after this section
• 4.2. References and readings
• 4.3. Covariance
• 4.4. Correlation
  • 4.4.1. The scatterplot matrix
  • 4.4.2. Visualizing the correlation matrix
  • 4.4.3. What does a near-zero correlation look like?
• 4.5. Some definitions
• 4.6. Least squares models with a single x-variable
  • 4.6.1. Minimizing errors as an objective
  • 4.6.2. Solving the least squares problem and interpreting the model
  • 4.6.3. Example
  • 4.6.4. A larger example: predicting vapour pressure
• 4.7. Least squares model analysis
  • 4.7.1. The variance breakdown
  • 4.7.2. Confidence intervals for the model coefficients \(b_0\) and \(b_1\)
  • 4.7.3. Prediction error estimates for the y-variable
  • 4.7.4. Interpretation of software output
• 4.8. Investigating an existing linear model
  • 4.8.1. Summary so far
  • 4.8.2. The assumption of normally distributed errors
  • 4.8.3. Non-constant error variance
  • 4.8.4. Lack of independence in the data
  • 4.8.5. Linearity of the model (incorrect model specification)
• 4.9. Summary of steps to build and investigate a linear model
  • 4.9.1. A worked example of the workflow in Python
• 4.10. More than one variable: multiple linear regression (MLR)
  • 4.10.1. Multiple linear regression: notation
  • 4.10.2. Estimating the model parameters via optimization
  • 4.10.3. Interpretation of the model coefficients
  • 4.10.4. \(R^2\) never decreases as terms are added
  • 4.10.5. Integer (dummy, indicator) variables in the model
• 4.11. Outliers: discrepancy, leverage, and influence of the observations
  • 4.11.1. Background
  • 4.11.2. Leverage
  • 4.11.3. Discrepancy
  • 4.11.4. Influence
  • 4.11.5. Computing these diagnostics in Python
  • 4.11.6. The influence plot
• 4.12. Enrichment topics
  • 4.12.1. Nonparametric models
  • 4.12.2. Robust least squares models
  • 4.12.3. Logistic modelling (regression)
  • 4.12.4. Testing of least-squares models
  • 4.12.5. Bootstrapping
• 4.13. Exercises