Course: Linear models and their mathematical foundations (Lineare Modelle und ihre mathematischen Grundlagen)

Winter term 2018/19

Course description

Linear models play a central part in modern statistics. Linear models include, as special cases, simple and multiple linear regression, analysis of variance (factorial) models and linear mixed-effects models. These models share a number of properties, such as linearity, that can be exploited to good effect. These common properties enable us to study linear models as a single class, rather than as an unrelated collection of special topics. This course provides an intense, rigorous introduction into the broad spectrum of statistical linear models that is useful in the analysis of data as well as into the mathematical foundations underpinning such models. Students will need a solid background in calculus and matrix algebra. Statistical prerequisites include some exposure to statistical theory and to estimation and testing hypotheses. The following topics are covered by the course:

  • Aims and scope of linear models in statistics
  • Distributions of linear and quadratic functions
  • Simple linear regression
  • Multiple linear regression
  • Bayesian inference for linear regression
  • Shrinkage methods
  • Analysis of variance (ANOVA)
  • Analysis of covariance (ANCOVA)
  • Linear mixed-effects models
The lectures will be accompanied by classes covering both theoretical aspects and the practice of solving applied exercises using the software package R.

This course comprises 6 contact hours per week (4 hours of lectures, 2 hour class) and is compulsory for students enrolled in the Master of Applied Statistics programme at the Georg August University Göttingen.

Lecturer: PD Dr. Steffen Unkel

Class teacher: Cynthia Laurena Huber


  • Please use Stud.IP to sign up for this module so that we can send e-mails to all participants from there.


Class Wednesday 12:15 - 13:45 Seminar room VG 3.103, Verfügungsgebäude
Lecture Wednesday 14:15 - 15:45 Lecture hall ZHG104, Zentrales Hörsaalgebäude
Lecture Thursday 12:15 - 13:45 Lecture hall MED 23, Humboldtallee 32


Date Lecture/Class Topic Material Recommended reading
October 24th (Wednesday) Lecture Setting the scene Lecture slides

Initial data analysis, Rmd file

History of regression, Rmd file
October 25th (Thursday) Lecture Preliminaries Lecture slides, R code Rencher and Schaalje (2008),
Chapters 4 and 5
November 1st (Thursday) Lecture Preliminaries (continuation)
November 7th (Wednesday) Class Study sheet 1 Study sheet 1
November 7th (Wednesday) Lecture Simple linear regression Lecture slides SLR

Lecture slides MLE, R code

scores.txt, HTML file, Rmd file
Rencher and Schaalje (2008), Chapter 6, Section 10.1 and Section 10.2
Casella and Berger (2002), Section 11.3 and Section 12.2
November 8th (Thursday) Lecture Simple linear regression (continuation)
November 14th (Wednesday) Class Study sheet 2 Study sheet 2
November 14th (Wednesday) Lecture Multiple regression: Estimation Lecture slides

table71.txt, HTML file, Rmd file
Rencher and Schaalje (2008), Chapter 7
November 15th (Thursday) Lecture Multiple regression: Estimation (continuation)
November 21st (Wednesday) Class Study sheet 3 Study sheet 3
November 21st (Wednesday) Lecture Multiple regression: Tests of hypotheses and confidence intervals Lecture slides

Hypothesis testing: HTML file, Rmd file, table74.txt
Confidence intervals: HTML file, Rmd file
LR tests: HTML file, Rmd file
Rencher and Schaalje (2008), Chapter 8
November 22nd (Thursday) Lecture Multiple regression: Tests of hypotheses and confidence intervals (continuation)
November 28th (Wednesday) Class Study sheet 4 Study sheet 4, R code for exercise 2
November 28th (Wednesday) Lecture Multiple regression: Diagnostics Lecture slides

HTML file, Rmd file
Rencher and Schaalje (2008), Chapter 9
November 29th (Thursday) Lecture Multiple regression: Diagnostics (continuation)
December 5th (Wednesday) Class Study sheet 5 Study sheet 5, data1.txt, table75.txt
R code for exercise 4, R code for exercise 5, R code for exercise 6, R code for exercise 7
December 5th (Wednesday) Lecture Multiple regression: Bayesian inference Lecture slides

table111.txt, HTML file, Rmd file
Rencher and Schaalje (2008), Chapter 11
Fahrmeir et al. (2013): Section 4.4.1
December 6th (Thursday) Computer Lab Linear Regression with R Exercises, dbp.txt, R code
December 12th (Wednesday) Class Study sheet 6 Study sheet 6
R code for exercise 1, R code for exercise 5
December 13th (Thursday) Lecture Shrinkage regression Lecture slides PCA

Lecture slides Shrinkage regression, HTML file, Rmd file
Jolliffe (2002), Chapter 8
Faraway (2015): Section 11.1, 11.3 and 11.4
December 19th (Wednesday) Class Study sheet 7 Study sheet 7, lesen.txt

R code for exercise 1, R code for exercise 2, R code for exercise 4
December 19th (Wednesday) Lecture Shrinkage regression (continuation)
December 20th (Thursday) Computer Lab Cross-validation and shrinkage methods with R Exercises, R code
January 9th (Wednesday) Class Study Sheet 8 Study sheet 8, R code for exercise 4
January 9th (Wednesday) Lecture Factorial models Lecture slides

One way ANOVA: table132.txt, HTML file, Rmd file
Two-way ANOVA:table142.txt, HTML file, Rmd file
Rencher and Schaalje (2008),
Chapter 12
January 10th (Thursday) Lecture Factorial models (continuation)
January 16th (Wednesday) Class Study Sheet 9 Study sheet 9, table136.txt
R code for exercise 1, R code for exercise 2, R code for exercise 4
January 16th (Wednesday) Lecture Linear mixed-effects models Lecture slides, HTML file, Rmd file Fahrmeir et al. (2013), Section 7.1, 7.2, 7.3 and 7.7 (or Rencher and Schaalje (2008), Chapter 17)
January 17th (Thursday) Lecture Linear mixed-effects models (continuation)
January 23rd (Wednesday) Class Study Sheet 10 Study sheet 10, R code for exercise 4
January 23rd (Wednesday) Computer Lab ANOVA and Linear mixed-effects models with R Exercises, R code

Supplemental resources

Topic Material
Matrix AlgebraLecture slides and exercises from the course Mathematical Foundations of Applied Statistics
R package HistData Data Sets from the History of Statistics and Data Visualization, R package version 0.7-8.



  • Casella, G. and Berger, R. L. (2002): Statistical Inference, 2nd edition, Duxbury
  • Fahrmeir, L., Kneib. T., Lang, S. and Marx, B. (2013): Regression: Models, Methods and Applications, Springer
  • Hastie, T., Tibshirani, R., Friedman, J. (2009): The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition, Springer
  • Jolliffe, I. T. (2002): Principal Component Analysis, 2nd edition, Springer
  • Rencher, A. C. and Schaalje, G. B. (2008): Linear Models in Statistics, 2nd edition, Wiley
  • Searle, S. R. and Gruber, M. H. J. (2016): Linear Models, 2nd edition, Wiley
  • Gruber, M. H. J. (2014): Matrix Algebra for Linear Models, Wiley
  • Faraway, J. J. (2015): Linear Models with R, 2nd edition, Chapman & Hall/CRC
  • Faraway, J. J. (2016): Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models, 2nd Edition, Chapman & Hall/CRC


Written exam (duration: 90 minutes). The exam will take place on Thursday, January 31st 2019, between 12:15 p.m. and 13:45 p.m. in the lecture hall MED 23, Humboldtallee 32. The permitted examination aids are:

  • Unannotated lecture slides;
  • One handwritten DIN A4 sheet (one-sided);
  • Pocket calculator;
  • English dictionary (without personal comments).