0
$\begingroup$

For the simple linear regression model:

$Y_i = \beta*X_i + \epsilon_i$

I want to find CI for $\beta x$ which is $E(Y_i)$ when $x_i=x$

I find that $\hat\beta$ ~ $N(\beta, \frac{\sigma^2}{\sum(X_i^2)})$, so the distribution of $\hat\beta x$ is $N(\beta x$, $\frac{X^2\sigma^2}{\sum(X_i^2)})$. If the variance is known I can use $P(-z_{\alpha/2}$ $\leq$ $\frac{\hat\beta x - \beta x}{x\sigma/\sum(x_i^2)}$ $\leq$ $z_{\alpha/2})$.

If the variance is unknown, what unbiased estimator should I use for $\sigma^2$? Is it the sample variance? What is it in this case?

$\endgroup$
0
$\begingroup$

Suppose number of predictors is $p$ and number of observations is $N$. Let $X$ be $N\times p$ matrix, and the specific $x$ be $1\times p$. Then $$\frac{x\beta-x\hat{\beta}}{\hat{\sigma}\sqrt{x(X^TX)^{-1}x}} \sim t_{N-p}$$ where $$\hat{\sigma}^2 = \frac{RSS}{N-p}=\frac{(Y-X\hat{\beta})^T(Y-X\hat{\beta})}{N-p}.$$

In the above $t$ stands for t distribution.

For simple regression $p=1$, so $\hat{\sigma}^2$ turns out to be sample variance. However, you need to use $t$ distribution, not Gaussian.

$\endgroup$
  • $\begingroup$ So you're saying in this case it's still the sample variance? $\endgroup$ – Rainroad Dec 3 '16 at 11:32
  • $\begingroup$ Yes. But do not use $z_{\alpha/2}$ of normal distribution. Use t distribution. $\endgroup$ – galan Dec 3 '16 at 23:26
0
$\begingroup$

If your model is $Y_i = \beta x_i + \epsilon_i$, then the unbiased estimator for $var(\epsilon_i)=\sigma^2$ is given by $s^2 = \frac{\sum e_i^2}{n-1}$, where $e_i= y_i - \hat{y}_i$.

$\endgroup$
  • $\begingroup$ Can I use $s^2$ = $\frac {1}{n} \sum (Y_i - \beta X_i)^2$ $\endgroup$ – Rainroad Dec 4 '16 at 16:17
  • $\begingroup$ You can, but it will be biased estimator $\endgroup$ – V. Vancak Dec 4 '16 at 19:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.