Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some invertible, $p\times p$ positive definite matrix $\Sigma$.

Let $\hat{\mu}_0 := \frac{\sum_{i=1}^{n_1}x_i + \sum_{i=1}^{n_2}y_i}{n_1 + n_2}$, $\hat{\mu}_1 := \frac{1}{n_1}\sum_{i=1}^{n_1}x_i$ and $\hat{\mu}_2 := \frac{1}{n_2}\sum_{i=1}^{n_2}y_i$

Suppose $\hat{\Sigma}_0=\frac{1}{n_1+n_2}\biggl(\sum^{n_1}_{i=1}(x_i-\hat{\mu}_0)(x_i-\hat{\mu}_0)^T+\sum^{n_2}_{i=1}(y_i-\hat{\mu}_0)(y_i-\hat{\mu}_0)^T\biggr)$

and $\hat{\Sigma}=\frac{1}{n_1+n_2}\biggl(\sum^{n_1}_{i=1}(x_i-\hat{\mu}_1)(x_i-\hat{\mu}_1)^T+\sum^{n_2}_{i=1}(y_i-\hat{\mu}_2)(y_i-\hat{\mu}_2)^T\biggr)$

I would like to show that:

$$\frac{\det(\hat{\Sigma}_0)}{\det(\hat{\Sigma})} = 1 + \frac{n_1n_2}{(n_1+n_2)^2}(\hat{\mu}_1 -\hat{\mu}_2)^T\hat\Sigma^{-1}(\hat{\mu}_1 -\hat{\mu}_2) $$

I know that $\frac{\det(\hat{\Sigma}_0)}{\det(\hat{\Sigma})}=\det(\hat{\Sigma}^{-1/2}\hat{\Sigma}_0\hat{\Sigma}^{-1/2})$, but I'm not sure how to continue.

  • $\begingroup$ What is $\hat{\mu_0}$? $\endgroup$ – Benjamin Wang Jan 26 at 20:40
  • $\begingroup$ My guess is that $\hat{\mu}_0$ is meant to be the sample mean of the combined vector $(X_1, \ldots, X_{n_1}, Y_1, \ldots, Y_{n_2})$. $\endgroup$ – nullUser Jan 27 at 2:26
  • $\begingroup$ @BenjaminWang it's the mean under the null hypothesis $H_0:\mu_1=\mu_2$ $\endgroup$ – user634512 Jan 27 at 5:44
  • $\begingroup$ @nullUser no, see above $\endgroup$ – user634512 Jan 27 at 5:44
  • 3
    $\begingroup$ Please familiarize yourself with the rules on bounties. You lose the points when you start the bounty. Placing a bounty does not entitle you to an answer. Nor is an arrangement with another user possible. I am sure that Marina's answer will be deleted shortly, and they will lose the points. $\endgroup$ – Jyrki Lahtonen Jan 28 at 9:28

After fixing the missing $\hat{}$ over the $\Sigma$ on the right-hand side, the equation is trivial to prove via the Matrix Determinant Lemma. It tells us that $\det(A+uv^T) = (1+v^T A^{-1}u)\det(A)$, and we simply need to match the terms: With $A=\hat\Sigma$, $u=v=\frac{\sqrt{n_1 n_2}}{n_1 + n_2}(\hat \mu_1 -\hat\mu_2)$ it follows that

$$\begin{aligned} \Big(1 + u^T\hat\Sigma^{-1}v\Big)\det(\hat\Sigma) &= \det(\hat\Sigma + uv^T) \end{aligned}$$

Now, all we need to do is simplify until we arrive at $\hat \Sigma_0$. Using $\hat\mu_1 = \frac{1}{n_1} X^T 1$, $\mu_2=\frac{1}{n_2}Y^T1$ we have

$$\begin{aligned} \sum_{i=1}^{n_1} (x_i-\hat\mu_1)(x_i-\hat\mu_1)^T &= (X-1\hat\mu_1^T)^T(X-1\hat\mu_1^T) \\&= X^TX-\hat\mu_11^TX -X^T 1 \hat\mu_1 + \hat\mu_1 1^T1 \hat\mu_1^T \\&=\boxed{X^TX-n_1\hat\mu_1\hat\mu_1^T} \end{aligned}$$

And similarly $Y^TY-n_2\hat\mu_2\hat\mu_2^T$ for the $y$-sum. On the other hand

$$\begin{aligned} \sum_{i=1}^{n_1} (x_i-\hat\mu_0)(x_i-\hat\mu_0)^T &= (X-1\hat\mu_0^T)^T(X-1\hat\mu_0^T) \\&= X^TX-\hat\mu_01^TX -X^T 1 \hat\mu_0 + \hat\mu_0 1^T1 \hat\mu_0^T \\&=\boxed{X^TX - n_1\hat\mu_0\hat\mu_1^T - n_1\hat\mu_1\hat\mu_0 + n_1\hat\mu_0\hat\mu_0^T} \end{aligned}$$

And similarly $Y^TY - n_2\hat\mu_0\hat\mu_2^T - n_2\hat\mu_2\hat\mu_0 + n_2\hat\mu_0\hat\mu_0^T$ for the $y$-sum. Adding both together, and using $\hat \mu_0 = \frac{n_1 \hat\mu_1 + n_2\hat\mu_2}{n_1 + n_2}$ we find

$$\begin{aligned} &X^TX - n_1\hat\mu_0\hat\mu_1^T - n_1\hat\mu_1\hat\mu_0 + n_1\hat\mu_0\hat\mu_0^T \\&+Y^TY - n_2\hat\mu_0\hat\mu_2^T - n_2\hat\mu_2\hat\mu_0 + n_2\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-\hat\mu_0(n_1\mu_1^T + n_2\mu_2^T) - (n_1\mu_1 + n_2\mu_2)\hat\mu_0^T +(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-(n_1+n_2)\hat\mu_0\hat\mu_0^T - (n_1+n_2)\hat\mu_0\hat\mu_0^T +(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=\boxed{X^TX+Y^TY-\frac{(n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T}{n_1+n_2}} \end{aligned}$$

Thus, our goal of showing $\hat\Sigma_0 = \hat\Sigma +uv^T$ is equivalent to (after cancelling $X^TX$ and $Y^TY$ terms)

$$ -\frac{(n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T}{(n_1+n_2)^2} =\frac{-n_1\hat\mu_1\hat\mu_1^T-n_2\hat\mu_2\hat\mu_2^T}{n_1+n_2} + \frac{n_1n_2(\hat \mu_1 -\hat\mu_2)(\hat \mu_1 -\hat\mu_2)^T}{(n_1+n_2)^2} $$

Which is equivalent to

$$ (n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T + n_1n_2(\hat \mu_1 -\hat\mu_2)(\hat \mu_1 -\hat\mu_2)^T =(n_1+n_2)(n_1\hat\mu_1\hat\mu_1^T+n_2\hat\mu_2\hat\mu_2^T) $$

which is a true statement. q.e.d

Final Remark: Of course, the Matrix Determinant Lemma would also offer a strategy of figuring out the equation in the first place: If we wanted to find the relationship bewtween $\det(\hat\Sigma_0)$ and $\det(\hat\Sigma)$, it would suggest to express $\hat\Sigma_0 = \hat\Sigma + UV^T$. Since both $\hat\Sigma_0$ and $\hat\Sigma$ are symmetric, we are in fact guaranteed $\hat\Sigma_0 = \hat\Sigma + UU^T$ for some $(p\times k)$ matrix $U$ with $k\le p$ (generally we would choose the $U$ matrix with $k$ minimal)

  • $\begingroup$ Very nice @Hyperplane, very very nice. I learned a few things today, thank you. $\endgroup$ – A rural reader Jan 31 at 4:18
  • $\begingroup$ Thank you very much for the question. Would you mind taking a look at this question? math.stackexchange.com/questions/3718725/… $\endgroup$ – user634512 Jan 31 at 11:21
  • $\begingroup$ It is a nice solution. (+1) $\endgroup$ – River Li Feb 4 at 1:37

Python simulation showing that the right-hand side must contain $\hat\Sigma^{-1}$ (equation in comments), not $\Sigma^{-1}$ (your original equation)

import numpy as np
from numpy.linalg import det, inv
from scipy.stats import invwishart, multivariate_normal as normal

n1, n2, p = 10, 12, 5
μ1, μ2 = normal.rvs(size=(2,p))
Σ  = invwishart(df=p+1, scale=np.eye(p)).rvs()
X  = normal(mean=μ1, cov=Σ).rvs(n1)
Y  = normal(mean=μ2, cov=Σ).rvs(n2)

μ0hat = (X.sum(axis=0) + Y.sum(axis=0))/(n1+n2)
μ1hat = X.mean(axis=0)
μ2hat = Y.mean(axis=0)

Σ0hat  = ((X-μ0hat).T@(X-μ0hat) + (Y-μ0hat).T@(Y-μ0hat))/(n1+n2)
Σhat= ((X-μ1hat).T@(X-μ1hat) + (Y-μ2hat).T@(Y-μ2hat))/(n1+n2)
LHS = det(Σ0hat)/det(Σhat)
RHS = 1 +  ((n1*n2)/(n1+n2)**2) * (μ1hat-μ2hat)@inv(Σ)@(μ1hat -μ2hat)
corrected_RHS = 1 +  ((n1*n2)/(n1+n2)**2) * (μ1hat-μ2hat)@inv(Σhat)@(μ1hat -μ2hat)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy