In regression analysis for classical linear regression model the residual term is independent of x and y and normally distributed and it is a random variable but i found somewhere written u~N and u~NID.I cannot understand the difference so can someone explain the meaning of NID (normally and independently distributed)?(sorry for my bad English).


In linear regression with Gaussian (and heteroscedastic) noise, our model assumes that for $n$ observations of data, for each $i \in [n]$,

$$Y_i = \beta X_i + \epsilon_i,$$

where $\epsilon_i$ is our ERROR term for the $i$th observation (note that residual $e_i$ is an estimator of $\epsilon_i$) Such that $\epsilon_i \sim N(0,\sigma^2_i).$ NID means "Gaussian and independently distributed", which is essentially a slightly more lenient way of saying that $\forall i \in [n],$ $\epsilon_i$ is independent of $\epsilon_j$, $j \neq i$ (i.e. errors are independent across observations).

Note that our residual $e_i$ is not necessarily independent of $Y_i$ depending on how we estimate $\epsilon_i$. Most of the time, our residuals are modeled as $e_i = \hat{Y}_i - Y_i,$ where $\hat{Y}_i$ is our prediction for the $i$th observation generated as

$$\hat{Y}_i = \hat{\beta}X_i.$$

In this case, $e_i$ is not independent of $Y_i$.

  • $\begingroup$ (I will just leave my comment here to add up to what is said. Sorry, I didn't expect you to edit your answer). Looks like a lot of papers use "normally and independently distributed" (sometimes adding - "with constant variance") in a sense on normal and i.i.d. However, I found clear evidence that it can be used to the model, where variance is not constant, so it is better to keep that in mind. $\endgroup$ – Slowpoke May 25 '16 at 20:26
  • $\begingroup$ I think your residual expression might be $e_i=Y_i-\hat{Y}_i$ $\endgroup$ – Henry May 26 '16 at 6:47

"In regression analysis for classical linear regression model the residual term is independent of x and y." The error term must be assumed to be independent of x if the regression is to be unbiased, but it cannot be independent of y since it is a stochastic driver of y (y = a + b*x + error). The assumption that the error term is normally distributed is in general NOT required, unless the sample is small and the researcher wishes to rely upon "exact" t-statistics.

Beware, the answer shown above is semi-informed gibberish, from someone who doesn't know what "heteroscedastic" means, nor how a regression residual is defined.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.