Foundations of Multiple Regression and Analysis of Variance

This book provides a rigorous development of the foundations of linear models for multiple regression and Analysis of Variance (ANOVA), based on orthogonal projections and relations among linear subspaces. It is appropriate for the linear models course required in most statistics Ph.D. programs.

The presentation is particularly accessible because it is self-contained, general, and taken in logical steps that are linked directly to practicable computations. The broad objective is to provide a path of mastery so that the reader could, if stranded on a desert isle with nothing but pencil, paper, and a computer to perform matrix sums and products, replicate general linear models procedures in extant statistical computing packages.

The primary prerequisite is mathematical maturity, which includes logical thinking and the ability to tell when a proof is a proof. Casual acquaintance with matrices would be helpful but not required. Background in basic statis- tical theory and methods is assumed, mainly for familiarity with terminology and the purposes of statistics in applications.

The material is developed as a series of propositions, each dependent only on those preceding it. The reader is strongly encouraged to prove each one independently. Mastery requires active involvement.

As part of the broad coverage of the mathematics supporting multiple regression and ANOVA, those propositions also establish several new, key results.

• There is a unique, best numerator sum of squares for testing an estimable function
• The extra residual sum of squares due to imposing a linear hypothesis tests exclusively the estimable part
• Models that include exclusively any given set of ANOVA e?ects can be formulated with contrast coding
• Tests of any ANOVA e?ects in any design and model, including unbalanced and empty cells, can be had with extra residual sum of squares due to deleting predictor variables
• Essential properties of Type III methods are identi?ed and proven