0
$\begingroup$

This is a bit of an embarrassing question because I thought I knew much (and have been using linear regression since forever) about regression until I read something in a text that challenged this.

For a linear regression curve Y on X, the mean of the distribution of the empirical $Y's$ is given by $\hat{Y} = \alpha + \beta x$. The $Y's$ have the form $Y = \alpha + \beta x + \epsilon$ where $\epsilon$ is (usually) standard normally distributed - arising from experimental variability.

Recall that for a set of $n$ observations/ empirical results demonstrating linearity, optimality of a regression curve fit to this set of observation is equivalent to obtaining the optimal coefficient $\alpha$, $\beta$ - and this is achieved by minimising the function

$\sum_{i=1}^{n}(y - \hat{y_{i}})^{2} = \sum_{i=1}^{n}(y - (\alpha + b x_{i}))^{2}$.

In a text, the author claims that $a, b$ are estimates of $\alpha, \beta$. I've never drawn any distinction between $a, b$ and $\alpha, beta$ prior to having read this.

Would be great if someone could shed some light on this.

$\endgroup$
1
$\begingroup$

$a$ and $b$ are the coefficients you obtain after performing linear regression on the data, which you can do regardless of what the data look like. in certain situations you may believe the data follow some linear relationship $Y = \alpha + \beta x + \epsilon$, where here $\alpha$ and $\beta$ are some "true" but unknown parameters that you would like to estimate. It is then prudent to ask whether the coefficients $a$ and $b$ you obtained from the linear regression are close to the "true" parameters $\alpha$ and $\beta$.

$\endgroup$
  • $\begingroup$ Helpful. $\alpha, \beta$ maybe thought of as a family of values generating a family of curves. $\endgroup$ – Mathematicing Oct 20 at 2:48
0
$\begingroup$

$\alpha$ and $\beta$ are "true" and unobservable values which describe the nature of relationship between $X$ and $Y$. As I mentioned, those are unobservable, however, you need to measure the relationship between $X$ and $Y$ somehow. Therefore, you try to estimate $\alpha$ and $\beta$. By applying, for instance OLS, you estimate $a$ for $\alpha$, and $b$ for $\beta$. According Gauss-Markov Theorem, if certain assumptions hold, then you can assume that $a$ and $b$ are the "best" (or closest) estimaters for $\alpha$ and $\beta$, respectively. But again, remember, you never observe "true" parameters, just you try to find "closest" estimaters $a$ and $b$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.