4
$\begingroup$

Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some invertible, $p\times p$ positive definite matrix $\Sigma$.

Let $\hat{\mu}_0 := \frac{\sum_{i=1}^{n_1}x_i + \sum_{i=1}^{n_2}y_i}{n_1 + n_2}$, $\hat{\mu}_1 := \frac{1}{n_1}\sum_{i=1}^{n_1}x_i$ and $\hat{\mu}_2 := \frac{1}{n_2}\sum_{i=1}^{n_2}y_i$

Suppose $\hat{\Sigma}_0=\frac{1}{n_1+n_2}\biggl(\sum^{n_1}_{i=1}(x_i-\hat{\mu}_0)(x_i-\hat{\mu}_0)^T+\sum^{n_2}_{i=1}(y_i-\hat{\mu}_0)(y_i-\hat{\mu}_0)^T\biggr)$

and $\hat{\Sigma}=\frac{1}{n_1+n_2}\biggl(\sum^{n_1}_{i=1}(x_i-\hat{\mu}_1)(x_i-\hat{\mu}_1)^T+\sum^{n_2}_{i=1}(y_i-\hat{\mu}_2)(y_i-\hat{\mu}_2)^T\biggr)$

I would like to show that:

$$\frac{\det(\hat{\Sigma}_0)}{\det(\hat{\Sigma})} = 1 + \frac{n_1n_2}{(n_1+n_2)^2}(\hat{\mu}_1 -\hat{\mu}_2)^T\hat\Sigma^{-1}(\hat{\mu}_1 -\hat{\mu}_2) $$

I know that $\frac{\det(\hat{\Sigma}_0)}{\det(\hat{\Sigma})}=\det(\hat{\Sigma}^{-1/2}\hat{\Sigma}_0\hat{\Sigma}^{-1/2})$, but I'm not sure how to continue.

$\endgroup$
11
  • $\begingroup$ What is $\hat{\mu_0}$? $\endgroup$ – Benjamin Wang Jan 26 at 20:40
  • $\begingroup$ My guess is that $\hat{\mu}_0$ is meant to be the sample mean of the combined vector $(X_1, \ldots, X_{n_1}, Y_1, \ldots, Y_{n_2})$. $\endgroup$ – nullUser Jan 27 at 2:26
  • $\begingroup$ @BenjaminWang it's the mean under the null hypothesis $H_0:\mu_1=\mu_2$ $\endgroup$ – user634512 Jan 27 at 5:44
  • $\begingroup$ @nullUser no, see above $\endgroup$ – user634512 Jan 27 at 5:44
  • 3
    $\begingroup$ Please familiarize yourself with the rules on bounties. You lose the points when you start the bounty. Placing a bounty does not entitle you to an answer. Nor is an arrangement with another user possible. I am sure that Marina's answer will be deleted shortly, and they will lose the points. $\endgroup$ – Jyrki Lahtonen Jan 28 at 9:28
4
+150
$\begingroup$

After fixing the missing $\hat{}$ over the $\Sigma$ on the right-hand side, the equation is trivial to prove via the Matrix Determinant Lemma. It tells us that $\det(A+uv^T) = (1+v^T A^{-1}u)\det(A)$, and we simply need to match the terms: With $A=\hat\Sigma$, $u=v=\frac{\sqrt{n_1 n_2}}{n_1 + n_2}(\hat \mu_1 -\hat\mu_2)$ it follows that

$$\begin{aligned} \Big(1 + u^T\hat\Sigma^{-1}v\Big)\det(\hat\Sigma) &= \det(\hat\Sigma + uv^T) \end{aligned}$$

Now, all we need to do is simplify until we arrive at $\hat \Sigma_0$. Using $\hat\mu_1 = \frac{1}{n_1} X^T 1$, $\mu_2=\frac{1}{n_2}Y^T1$ we have

$$\begin{aligned} \sum_{i=1}^{n_1} (x_i-\hat\mu_1)(x_i-\hat\mu_1)^T &= (X-1\hat\mu_1^T)^T(X-1\hat\mu_1^T) \\&= X^TX-\hat\mu_11^TX -X^T 1 \hat\mu_1 + \hat\mu_1 1^T1 \hat\mu_1^T \\&=\boxed{X^TX-n_1\hat\mu_1\hat\mu_1^T} \end{aligned}$$

And similarly $Y^TY-n_2\hat\mu_2\hat\mu_2^T$ for the $y$-sum. On the other hand

$$\begin{aligned} \sum_{i=1}^{n_1} (x_i-\hat\mu_0)(x_i-\hat\mu_0)^T &= (X-1\hat\mu_0^T)^T(X-1\hat\mu_0^T) \\&= X^TX-\hat\mu_01^TX -X^T 1 \hat\mu_0 + \hat\mu_0 1^T1 \hat\mu_0^T \\&=\boxed{X^TX - n_1\hat\mu_0\hat\mu_1^T - n_1\hat\mu_1\hat\mu_0 + n_1\hat\mu_0\hat\mu_0^T} \end{aligned}$$

And similarly $Y^TY - n_2\hat\mu_0\hat\mu_2^T - n_2\hat\mu_2\hat\mu_0 + n_2\hat\mu_0\hat\mu_0^T$ for the $y$-sum. Adding both together, and using $\hat \mu_0 = \frac{n_1 \hat\mu_1 + n_2\hat\mu_2}{n_1 + n_2}$ we find

$$\begin{aligned} &X^TX - n_1\hat\mu_0\hat\mu_1^T - n_1\hat\mu_1\hat\mu_0 + n_1\hat\mu_0\hat\mu_0^T \\&+Y^TY - n_2\hat\mu_0\hat\mu_2^T - n_2\hat\mu_2\hat\mu_0 + n_2\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-\hat\mu_0(n_1\mu_1^T + n_2\mu_2^T) - (n_1\mu_1 + n_2\mu_2)\hat\mu_0^T +(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-(n_1+n_2)\hat\mu_0\hat\mu_0^T - (n_1+n_2)\hat\mu_0\hat\mu_0^T +(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=\boxed{X^TX+Y^TY-\frac{(n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T}{n_1+n_2}} \end{aligned}$$

Thus, our goal of showing $\hat\Sigma_0 = \hat\Sigma +uv^T$ is equivalent to (after cancelling $X^TX$ and $Y^TY$ terms)

$$ -\frac{(n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T}{(n_1+n_2)^2} =\frac{-n_1\hat\mu_1\hat\mu_1^T-n_2\hat\mu_2\hat\mu_2^T}{n_1+n_2} + \frac{n_1n_2(\hat \mu_1 -\hat\mu_2)(\hat \mu_1 -\hat\mu_2)^T}{(n_1+n_2)^2} $$

Which is equivalent to

$$ (n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T + n_1n_2(\hat \mu_1 -\hat\mu_2)(\hat \mu_1 -\hat\mu_2)^T =(n_1+n_2)(n_1\hat\mu_1\hat\mu_1^T+n_2\hat\mu_2\hat\mu_2^T) $$

which is a true statement. q.e.d


Final Remark: Of course, the Matrix Determinant Lemma would also offer a strategy of figuring out the equation in the first place: If we wanted to find the relationship bewtween $\det(\hat\Sigma_0)$ and $\det(\hat\Sigma)$, it would suggest to express $\hat\Sigma_0 = \hat\Sigma + UV^T$. Since both $\hat\Sigma_0$ and $\hat\Sigma$ are symmetric, we are in fact guaranteed $\hat\Sigma_0 = \hat\Sigma + UU^T$ for some $(p\times k)$ matrix $U$ with $k\le p$ (generally we would choose the $U$ matrix with $k$ minimal)

$\endgroup$
3
  • $\begingroup$ Very nice @Hyperplane, very very nice. I learned a few things today, thank you. $\endgroup$ – A rural reader Jan 31 at 4:18
  • $\begingroup$ Thank you very much for the question. Would you mind taking a look at this question? math.stackexchange.com/questions/3718725/… $\endgroup$ – user634512 Jan 31 at 11:21
  • $\begingroup$ It is a nice solution. (+1) $\endgroup$ – River Li Feb 4 at 1:37
1
$\begingroup$

Python simulation showing that the right-hand side must contain $\hat\Sigma^{-1}$ (equation in comments), not $\Sigma^{-1}$ (your original equation)

import numpy as np
from numpy.linalg import det, inv
from scipy.stats import invwishart, multivariate_normal as normal

n1, n2, p = 10, 12, 5
μ1, μ2 = normal.rvs(size=(2,p))
Σ  = invwishart(df=p+1, scale=np.eye(p)).rvs()
X  = normal(mean=μ1, cov=Σ).rvs(n1)
Y  = normal(mean=μ2, cov=Σ).rvs(n2)

μ0hat = (X.sum(axis=0) + Y.sum(axis=0))/(n1+n2)
μ1hat = X.mean(axis=0)
μ2hat = Y.mean(axis=0)

Σ0hat  = ((X-μ0hat).T@(X-μ0hat) + (Y-μ0hat).T@(Y-μ0hat))/(n1+n2)
Σhat= ((X-μ1hat).T@(X-μ1hat) + (Y-μ2hat).T@(Y-μ2hat))/(n1+n2)
LHS = det(Σ0hat)/det(Σhat)
RHS = 1 +  ((n1*n2)/(n1+n2)**2) * (μ1hat-μ2hat)@inv(Σ)@(μ1hat -μ2hat)
print(LHS-RHS)
corrected_RHS = 1 +  ((n1*n2)/(n1+n2)**2) * (μ1hat-μ2hat)@inv(Σhat)@(μ1hat -μ2hat)
print(LHS-corrected_RHS)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.