0
$\begingroup$

For a linear regression problem, $$ y = x_1\beta_1 + x_2\beta_2 + x_3\beta_3 + x_4\beta_4 + b + \epsilon $$ If I have method 1, that estimate the coefficients as: $$ \beta_{1,1}, \beta_{2,1}, \beta_{3,1}, \beta_{4,1} $$ with corresponding p-values: $$ p_{1,1}, p_{2,1}, p_{3,1}, p_{4,1} $$

With another method, I estimated: $$ \beta_{1,2}, \beta_{2,2}, \beta_{3,2}, \beta_{4,2} $$ with corresponding p-values: $$ p_{1,2}, p_{2,2}, p_{3,2}, p_{4,2} $$

If I have $|\beta_{1,1}-\beta_1|<|\beta_{1,2}-\beta_1|$ (i.e. method 1 can estimate $\beta_1$ with smaller bias). Does that tell me anything about $p_{1,1}$ and $p_{1,2}$?

What if we know for sure that $\beta_1\neq 0$? (but the null hypothesis will still be $\beta_1=0$)

Is there any general relationship between the estimation of $\beta$ and the p-value? Is it just the higher estimated $|\beta|$ is, the lower the p-value is (in general)?

$\endgroup$

1 Answer 1

0
$\begingroup$

It depends also on their (estimated) variance, i.e., if $|\beta_{1,1}| \le |\beta_{1,2}|$, and if their estimated variance is the same, thus $p_{1,2} \ge p_{1,1}$. However, without knowing anything on the variance - it is impossible to say nothing about this relationship. Generally, recall that the p.value is defined in the following way $$ p.v = P_{H_0}\left(T_{(n-p)}\ge\left|\frac{\hat{\beta}}{\hat{\sigma}_{\hat{\beta}} } \right| \right), $$ where $p$ is the number of $\beta$s, $T_{(n-p)}$ is a random variable that follows $t$-student distribution with $n-p$ degrees and the original data $\{y_i, x_{11},..., x_{1p}\}_{i=1}^n$ were generated by $Y+X\beta+\varepsilon$, where $$ \varepsilon\sim N_{n}(0,\sigma^2I). $$

$\endgroup$
5
  • $\begingroup$ Thanks. Let's assume they have the same variance if needed. However, I still have a big confusion. I think the testing method is usually testing against null hypothesis where $\beta=0$. The test statistics is related to $|\hat{\beta}-0|$ divided by variance. Why the small bias $\hat{\beta}-0$ indicates bigger $|\hat{\beta}-0|$? (or bigger $|\hat{\beta}-0|/\sigma_{\hat{\beta}}$ if we don't assume the variance is the same?) $\endgroup$
    – user134111
    Commented Jul 23, 2017 at 14:33
  • $\begingroup$ I'm not sure that we are talking in the same terms. What mean "small bias" in your context? $\endgroup$
    – V. Vancak
    Commented Jul 23, 2017 at 21:22
  • $\begingroup$ The bias I have in mind is $|\hat{\beta}-\beta|$, where $\hat{\beta}$ is the estimated effect sizes and $\beta$ is the effect size of the real generating process. Then "smaller bias" means the model calculates $\hat{\beta}$ better. $\endgroup$
    – user134111
    Commented Jul 23, 2017 at 21:27
  • $\begingroup$ and I just realize that I made a mistake in my first comment, the last sentence should be: "Why the small bias $|\hat{\beta}-\beta|$ indicates bigger $|\hat{\beta}-0|$? (or bigger $|\hat{\beta}-0|/\sigma_\beta$ if we don't assume the variance is the same?)". Thanks. $\endgroup$
    – user134111
    Commented Jul 23, 2017 at 21:29
  • $\begingroup$ You are right. I've edited the answer. Basically, the p.value is calculated w.r.t, to the null hypothesis that is $\beta = 0$. The actual value of $\beta$ does not plays a role in the calculations. $\endgroup$
    – V. Vancak
    Commented Jul 23, 2017 at 21:37

You must log in to answer this question.