The standard error of estimate is the standard deviation of the prediction errors. It is computed like any other standard deviation - the square root of the error sum of squares divided by the degrees of freedom.
To compute the standard error of estimate, we begin by computing the error sum of squares (SSE) -
The next step is to compute the variance error (sE2) -
sE2 = ——
n - 2
The keen-eyed (or obsessive) among you will have noticed that the degrees of freedom are n-2 rather than n-1. The reason we subtract two in this instance is that variance error (and standard error of estimate) are statistics describing characteristics of two variables. They deal with the error involved in the prediction of Y (one variable) from X (the other variable). [I have also heard it said that the degrees of freedom are lost because we are estimating two constants in the prediction equation - intercept (A) and slope (B). Take your choice.]
By now, you know what the next equation is going to show. Yes, that is right! This one shows the derivation of the standard error of estimate from the variance error -
The standard error of estimation is interpreted in the same way as the standard deviation. The standard deviation tells us how spread out a distribution of scores is with respect to the distribution mean. The standard error of estimate tells us how spread out scores are with respect to their predicted values. If the error scores (E = Y - YPred) are normally distributed around the prediction line, about 68% of actual scores will fall between ±1 sE of their predicted values.
This is the second standard error we have discussed. You recall that the first one was the standard error of the mean which estimates the error in predicting a population mean from a sample mean. The standard error of estimation is used to assess the error of prediction for a regression line. There is a third standard error that is a critical feature of measurement theory - the standard error of measurement. We shall not be discussing it in this review, but for purposes of information, it is an enumeration of the error associated with estimating a true score from an obtained score.
Let us consider an example of the use of the standard error of estimation. We might be evaluating the extent to which duration of clinical supervision affects social worker performance. So, we collect data on the number of hours of clinical supervision received by each of a group of workers. We also randomly select three cases handled by each of the workers. These cases are evaluated by experienced social workers on a structured rating scale. Ratings may range from 0 to 100 with higher ratings representing more competent work. The cases with the highest and lowest ratings are dropped for each worker. The rating for the case with the middle rating becomes the worker's competency score.
Data were collected and analyzed for 20 workers with the following results -
We see that there is a positive relationship between length of supervision and competency rating. However, we don't know how accurately we can predict competency rating from supervision. We decide to compute the standard error of estimate. To do so, we first must calculate the variance error -
sE2 = ———— = 137.262
Now we take the square root of the variance error to get the standard error of estimate -
The standard error of 11.72 is fairly substantial. The following chart shows the prediction line and obtained ratings for the data we used to calulate this standard error of estimate. We can see the positive relationship between length of supervision and competency rating. Still, in a number of cases, the predicted competency rating differs substantially from the actual (obtained) rating.
This next chart shows another prediction line and ratings from a different group of workers. The prediction line for competency rating is about the same as for the first group, but the standard error of estimate is 5.86. As would be expected from the lower standard error, the actual ratings tend to cluster more closely around the prediction line; the prediction of competency rating from length of supervision is more accurate in this case than the previous one.
|©1999 by J.T. (Tim) Stocks, Ph.D.