OLS and what the robust errors are forOLS আর রোবাস্ট ত্রুটি কীসের জন্য

A regression in plain words

Put rainfall along the bottom and yield up the side, one dot per farm. Ordinary least squares picks the intercept and the slope of the line whose vertical gaps to the dots, squared and added up, are as small as they can be. "Ordinary" because every dot counts the same. The model is yield equals intercept plus slope times rainfall plus noise, and the noise is everything about a farm the line does not know: soil, seed, the farmer.

The slope is the coefficient, and it is a sentence: one more centimetre of rain goes with this much more yield, in the units the columns were typed in. Change rainfall to millimetres and the slope divides by ten while meaning exactly the same thing.

The four assumptions, in plain words

1. The relationship is a straight line, or you have transformed the columns until it is. A curve fitted with a line gives a slope that is right nowhere.
2. The noise averages zero at every level of rainfall. Nothing hiding in the noise also moves rainfall. If wetter villages also have richer soil, the slope carries the soil's credit and calls it rain.
3. The noise is equally spread at every level of rainfall. This is homoskedasticity, and when it fails the failure is called heteroskedasticity.
4. One farm's noise says nothing about another farm's. Independence.

The first two decide whether the slope is right. The last two decide whether the standard error is right. Robust errors mend the last two; nothing about the errors can mend the first two.

Worked example: six farms

The slope is the sum of the products divided by the sum of the squared rain deviations: 420 divided by 7,000, which is 0.06 maunds per centimetre. The intercept is the mean yield minus the slope times the mean rain: 16 minus 0.06 times 130, which is 16 minus 7.8, which is 8.2. The line is yield equals 8.2 plus 0.06 times rain. In words: across these six farms, ten centimetres more rain went with 0.6 of a maund more per bigha.

Where the standard error comes from

Put each farm's rain into the line and see what it predicts: 13, 14.2, 15.4, 16.6, 17.8 and 19. The gaps between what happened and what the line says are the residuals: 1, -2.2, 1.6, -1.6, 2.2 and -1. Square them and add: 1 plus 4.84 plus 2.56 plus 2.56 plus 4.84 plus 1 is 16.8. Divide by the observations minus the two things estimated, 6 minus 2, to get the noise variance: 4.2. The standard error of the slope is the square root of that divided by the sum of squared rain deviations: the square root of 4.2 over 7,000, which is the square root of 0.0006, which is 0.0245.

The t statistic is the slope over its standard error: 0.06 divided by 0.0245, which is 2.45. A t of about 2.4 says the slope sits two and a half standard errors away from zero. What that is worth depends on how many farms there are. With six, the line for a five per cent two-sided test is 2.78, so these six do not clear it. With thirty or more the line is about 2.04 and the same t clears it. And t says nothing about size: whether 0.06 of a maund per centimetre matters to a farmer is a question about farming, not about statistics.

Heteroskedasticity, and what HC1 does

The classical standard error above used one number, 4.2, for the spread of the noise at every level of rain. Suppose the big, wet farms are also the erratic ones. Then the spread is larger exactly where the rain deviations are largest, which is where the points pull hardest on the slope, and the classical standard error is too small: the slope is less certain than it says.

HC1 (heteroskedasticity-consistent, the first small-sample correction) does not assume one spread. It takes each farm's own squared residual, weights it by that farm's squared rain deviation, adds them up, divides by the squared sum of rain deviations, and multiplies by n over n minus k to make up for small samples. On the six farms: the weighted squares are 2,500, 4,356, 256, 256, 4,356 and 2,500, which add to 14,224. Times 6 over 4 is 21,336. Divide by 7,000 squared, 49,000,000, and take the square root: 0.0209. The t becomes 2.87.

The slope is still 0.06. It has to be: least squares chose it by minimising the gaps and never looked at how the gaps were spread. Robust errors re-estimate how sure you may be, never what you found. Six farms are too few for the robust number to mean much on its own; the point is which number moved.

Clustered errors

Sixty farms in six villages are not sixty independent facts about rain. Farms in one village share the weather, the soil, the same extension officer, so their residuals move together, and the fourth assumption fails by construction. Cluster-robust errors let the residuals correlate however they like inside a village and assume nothing between villages. Cluster at the level where the shock or the treatment varies: a village-level programme means village clusters even when the outcome is measured per farm.

Where it goes wrong

• Reporting the slope with no units, so nobody can tell whether 0.06 is a lot.
• Reading a large t as a large effect.
• Reaching for robust errors to rescue a slope that is biased by the second assumption. They cannot.
• Clustering at the farm level when the programme was assigned by village.
• Reporting HC1 errors for a sample of twenty as though they were exact.

Checklist

• The units of every coefficient are stated in the sentence that reports it.
• The scatter has been looked at before the line was fitted.
• Something has been said about what might sit in the noise and move the regressor too.
• A test for unequal spread has been run, or robust errors are reported by default.
• If observations share a group, errors are clustered on it and the number of clusters is stated.
• Effect size and t are reported as two different things.

The lab runs OLS with classical, HC1 or clustered errors from the method list and writes the APA table; the workshop's "which test" tool asks whether a regression is what your question needs, and its sample size tool says how many farms would let a slope this size clear the line.