Use and Abuse of Regression
George E. P. Box · Technometrics · 1966
where the O's are unknown parameters, the x's known constants, and the c's random variables uncorrelated and having the same variance and zero expectation, then estimates bo , b1 , .., bk of the S3's obtained by minimizing E (y -_ )2 with y = boxo + b 2x~ + b6x2 + * * * + bkXk are unbiassed and have smallest variance among all linear unbiassed estimates. The method of least squares is used in the analysis of data from planned experiments and also in the analysis of data from unplanned happenings. The word regression is most often used to describe analysis of unplanned data. It is the tacit assumption that the requirements for the validity of least squares analysis are satisfied for unplanned data that produces a great deal of trouble. Whether the data are planned or unplanned the quantity c, which is usually quickly dismissed as a random variable having the very specific properties mentioned above, really describes the effect of a large number of variables Xk+l , X+2 , ... , Xm which we know nothing about. If we suppose that it is enough to consider the linear effects of these latent variables (which would often be realistic for small variations in xk+ , -.. , x~) we should have