Goodness of Fit

How do we quantify the quality of fit?

The sum of squares of residuals ($S_r$) is a quantification of the error between the measured and predicted $y$ values after regression:

$$S_r=\sum _{i=1}^n(y_i-a_0-a_1{x}_i{)}^2$$

The total sum of squares around the mean value is the magnitude of the residual error associated with the dependent variable prior to regression:

$$S_t=\sum _{i=1}^n(y_i-\bar{y}{)}^2$$

We can then use then use the difference between $S_t$ and $S_r$ to quantify the improvement or error reduction. To do this, we define:

$$\begin{array}{rl}\text{Coefficient of Determination: }& r^2=\frac{S_t-S_r}{S_t}\\ \text{Correlation Coefficient: }& r\end{array}$$

If we have $r^2=1$, that means that the line explains all of the variability of the data and is therefore a “perfect fit”. On the other hand, $r^2=0$ would be a poor fit.

Something like $r^2=0.868$ means that 86% of the uncertainty is explained by the linear model.

It’s still important to check results visually, as $r^2$ and $r$ can trick you!