Linear regression analyzes a (linear) relationship between two variables or vectors, X and Y. Thus this relationship can in particular be described through a straight line. The equation for such is line is y = mx + b. The goal of linear regression is to adapt the values of the slope m, and of the intercept b, of the line so that the line gives the best prediction of Y from X. This is achieved by minimizing the sum of the squares of the vertical distances of the points from the line (S).
Thus the term
has to be minimized.
The two partial derivatives:
and
yield the following solutions for m and b:
The slope m of the line is given by:
and the intercept a is given by:
An example applet for three vectors is given here:
Ulrike Burkard, Dec. 17., 2001