Linear regression analyzes a (linear) relationship between two variables or
vectors, X and Y. Thus this relationship can in particular be
described through a straight line. The equation for such is line is y = mx
The goal of linear regression is to adapt the values of the slope m,
and of the intercept b, of the line so that the line gives the best prediction
of Y from X. This is achieved by minimizing the sum of the squares
of the vertical distances of the points from the line (S).
Thus the term
has to be minimized.
The two partial derivatives:
yield the following solutions for m and b:
The slope m of the line is given by:
and the intercept a is given by:
An example applet for three vectors is given here:
Ulrike Burkard, Dec. 17., 2001