Model: $$Y_{N\times 1} = X_{N\times M} \beta_{M\times 1} + \epsilon_{N\times 1}$$ the design matrix $X_{N\times M}$ is known, each column of the design matrix has been standardized to have mean 0 and variance 1. $Y$ is also known and has been standardized to mean 0 and variance 1. Also assume $\beta \sim \mathcal{N}(0, \sigma^2 I_{M\times M})$ and $\epsilon \sim \mathcal{N}(0, \sigma_\epsilon^2 I_{N\times N})$. In addition $M \sigma^2 + \sigma_\epsilon^2 = 1$

The marginal effect sizes are $\hat{\beta} = \frac{1}{N}X^\top Y$. The distribution the marginal effect sizes $\hat{\beta}$ can be determined as follows: \begin{align*} \hat{\beta} & = \frac{X^\top Y}{N} \\ & = \frac{X^\top ( X \beta + \epsilon) }{N} \\ & = \frac{XX^\top}{N} \beta + \frac{1}{N}X^\top \epsilon \end{align*} $$E[\hat{\beta} | X, \beta] = \frac{XX^\top}{N} \beta$$ $$\text{Var}[\hat{\beta} | X,\beta] = \frac{1}{N^2}X^\top E[\epsilon \epsilon^\top]X=\frac{1}{N^2}\sigma_\epsilon^2X^\top X$$ Thus $$\hat{\beta} \sim \mathcal{N}(\frac{XX^\top}{N} \beta, \frac{1}{N^2}\sigma_\epsilon^2X^\top X)$$ My question is that if we consider a small chunk of variable of size b, where the corresponding design matrix is of shape $X_{N \times b} $, the corresponding marginal effect sizes is $\hat{\beta}_b$: $$\hat{\beta}_b = \frac{X^\top_b Y}{N}$$ How is the following two derived: $$E[\hat{\beta}_b | \beta_b, X_b] = \frac{X_b^\top X_b}{N} \beta_b$$ $$\text{Var}[\hat{\beta}_b | \beta_b, X_b] = \frac{1}{N}( 1 - b \sigma^2) \frac{X_b^\top X_b}{N}$$

One thing I am confused about is the $X_b$ being conditioned in $E[\hat{\beta}_b | \beta_b, X_b]$. Does this mean that we don't assume anything on the rest of the design matrix? For more context, this is from the methods section in this paper.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.