Linear Regression

‘The basic formula:

The training data:

Objective function:

In here, is the mean squared error.

We get the optimal weight when we differentiate w.r.t. . Let’s expand first.

Next, we have to use Standard Results of Matrix Calculus, equation (4) and (1).

We set .

Note that the last line is only applicable if is invertible.

To determine the fitness of the model, we have R2 Goodness of Fit.