Statistics/Mathematics 3340 - Regression Analysis, Fall, 2025
Review materials:
Please review this material yourself as needed.
Selinger, Matrix Theory and Linear Algebra. material on projections in Chapter 5 will be useful.
James et al., Introduction to
Statistical Learning. Chapter 3 has material on simple and
multiple linear regression. Chapter 6 has material on model selection,
and chapter 7 discusses polynomial regression and splines.
Info on installation of R:
Notes on the installation of R, Rstudio and Rmarkdown.
Rmd file corresponding to the previous pdf.
LECTURE NOTES (under construction)
Topic 1: Simple linear regression -
notes from Stat2080, with a bit of added R code.
R code for simple linear regression. Illustrates the use of R as a calculator, with applicaton to simple linear regression.
Topic 2: Linear algebra
Some basic matrix algebra. Review on your own..
Projections. Includes derivation of prediction equation using linear algebra.
Topic 3: Intro to multiple regression in R
Using the "lm" command to carry out regression in R.
R commands to read data in a .csv file, and carry out a multiple regression using the "lm" function in R
Topic 4: Some useful regression models
Indicator variables, and their use in
analysis of variance models.
Types of linear regression models.
Polynomial regression.
Logistic regression.
Poisson regression.
Topic 5: Comparing models with the partial F test
Partial F test
example: carrying out the partial F test in R by comparing a full and a reduced model
Topic 6: Residual analysis
Intro to residual analysis
Common issues with residual plots
Topic 7: Differentiating with respect to a vector
Formulas for differentiating with respect to a vector. This gives a convenient method for deriving the least squares estimator in multiple regression.
(To deive the formulas, one needs some ideas from multivariable calculus, and a more advanced class
in multivariate statistical analysis such as Stat4350.)
Topic 8: Least squares estimation for the multiple regression model.
Multiple regression model, least squares estimation.
Derivation of the estimates of
intercept and slope for simple linear regression, using matrix calculations
Topic 9: Means and covariances, random vectors
Rules for expected values and variances of linear combinations of r.v.'s
Random vectors: definitions, expectation and covariance, including linear combinations.
R code to check mean and covariance calculations on random vectors page.
Topic 10: Sampling distributions and confidence intervals
sampling distributions of y, betahat, predicted values, residuals, and confidence intervals..
R code for construction of simulateous CI and an elliptical confidence region.
Topic 11: F tests
Cochran's theorem and the overall F test of significance.
review of hypothesis testing in multiple linear regression
Several examples showing how to test hypotheses using the lm and anova commands.
example: carrying out the partial F test in R - cement data set
an example
justification of the partial F test. The added variable plot
example of constructing an added variable plot
Testing the General Linear Hypothesis (section 3.3.4).
Diagnostics
leverage ( Chapter 6)
Multicollinearity (Chapters 3&9)
standardized residuals (Chapter 4) and case deletion statistics (Chapter 6)
transformations (chapter 5)
some linearizing transformations (table 5.4)
Model selection
Model selection. (Chapter 10)
ASSIGNMENTS
Assignment 1, due Thursday, September 12, 11:59 PM
This is the pdf version of the assignment.
n
R markdown file for assignment 1.
Assignment 1 solutions.
nnn
Assignment 2, due Sunday, September 22, 11:59 PM
This is the pdf version of the assignment.
R markdown file for assignment 2.
Assignment 2 solutions.
Assignment 3, due Sunday, October 6, 11:59 PM
Assignment 3 solutions.
Assignment 4, due Monday, October 28, 11:59 PM
Rmd file for assignment 4.
Assignment 4 solutions.
Assignment 5, due Monday, November 25, 11:59 PM
This is the pdf version of the assignment.
R markdown file for assignment 5.
assignment 5 solutions.
Assignment 6, due Wednesday, December 4, 11:59 PM
Assignment 6 solutions
Exam Info
The midterm exam will be on Tuesday, October 15, in class time.
midterm practice questions.
solutions to practice questions.
solutions to 2024 midterm
The final exam will be on Tuesday, December 10, 8:30-11:30 AM, in Dalplex.
Practice final examination .
This is the exam from fall, 2015. Coverage of this years exam may
vary somewhat, and will include some questions requiring derivations.
solutions to practice final examination
Formula sheet for final exam Formula sheet which will be included with exams - first ywo pages for midterm exam.
Statistical Tables