# What is $\frac{\det(\hat{\Sigma}_0)}{\det(\hat{\Sigma})}$ in terms of $\hat{\mu}_1$, $\hat{\mu}_2$ and $\hat{\Sigma}$

Let $$X_1,...,X_{n_1}$$ be an i.i.d. sample from $$N_p(\mu_1,\Sigma)$$ and let $$Y_1,...,Y_{n_2}$$ be an independent sample from $$N_p(\mu_2,\Sigma)$$, for some $$\mu_1,\mu_2 \in \mathbb{R}^p$$ and some invertible, $$p\times p$$ positive definite matrix $$\Sigma$$.

Let $$\hat{\mu}_0 := \frac{\sum_{i=1}^{n_1}x_i + \sum_{i=1}^{n_2}y_i}{n_1 + n_2}$$, $$\hat{\mu}_1 := \frac{1}{n_1}\sum_{i=1}^{n_1}x_i$$ and $$\hat{\mu}_2 := \frac{1}{n_2}\sum_{i=1}^{n_2}y_i$$

Suppose $$\hat{\Sigma}_0=\frac{1}{n_1+n_2}\biggl(\sum^{n_1}_{i=1}(x_i-\hat{\mu}_0)(x_i-\hat{\mu}_0)^T+\sum^{n_2}_{i=1}(y_i-\hat{\mu}_0)(y_i-\hat{\mu}_0)^T\biggr)$$

and $$\hat{\Sigma}=\frac{1}{n_1+n_2}\biggl(\sum^{n_1}_{i=1}(x_i-\hat{\mu}_1)(x_i-\hat{\mu}_1)^T+\sum^{n_2}_{i=1}(y_i-\hat{\mu}_2)(y_i-\hat{\mu}_2)^T\biggr)$$

I would like to show that:

$$\frac{\det(\hat{\Sigma}_0)}{\det(\hat{\Sigma})} = 1 + \frac{n_1n_2}{(n_1+n_2)^2}(\hat{\mu}_1 -\hat{\mu}_2)^T\hat\Sigma^{-1}(\hat{\mu}_1 -\hat{\mu}_2)$$

I know that $$\frac{\det(\hat{\Sigma}_0)}{\det(\hat{\Sigma})}=\det(\hat{\Sigma}^{-1/2}\hat{\Sigma}_0\hat{\Sigma}^{-1/2})$$, but I'm not sure how to continue.

• What is $\hat{\mu_0}$? – Benjamin Wang Jan 26 at 20:40
• My guess is that $\hat{\mu}_0$ is meant to be the sample mean of the combined vector $(X_1, \ldots, X_{n_1}, Y_1, \ldots, Y_{n_2})$. – nullUser Jan 27 at 2:26
• @BenjaminWang it's the mean under the null hypothesis $H_0:\mu_1=\mu_2$ – user634512 Jan 27 at 5:44
• @nullUser no, see above – user634512 Jan 27 at 5:44
• Please familiarize yourself with the rules on bounties. You lose the points when you start the bounty. Placing a bounty does not entitle you to an answer. Nor is an arrangement with another user possible. I am sure that Marina's answer will be deleted shortly, and they will lose the points. – Jyrki Lahtonen Jan 28 at 9:28

After fixing the missing $\hat{}$ over the $$\Sigma$$ on the right-hand side, the equation is trivial to prove via the Matrix Determinant Lemma. It tells us that $$\det(A+uv^T) = (1+v^T A^{-1}u)\det(A)$$, and we simply need to match the terms: With $$A=\hat\Sigma$$, $$u=v=\frac{\sqrt{n_1 n_2}}{n_1 + n_2}(\hat \mu_1 -\hat\mu_2)$$ it follows that

\begin{aligned} \Big(1 + u^T\hat\Sigma^{-1}v\Big)\det(\hat\Sigma) &= \det(\hat\Sigma + uv^T) \end{aligned}

Now, all we need to do is simplify until we arrive at $$\hat \Sigma_0$$. Using $$\hat\mu_1 = \frac{1}{n_1} X^T 1$$, $$\mu_2=\frac{1}{n_2}Y^T1$$ we have

\begin{aligned} \sum_{i=1}^{n_1} (x_i-\hat\mu_1)(x_i-\hat\mu_1)^T &= (X-1\hat\mu_1^T)^T(X-1\hat\mu_1^T) \\&= X^TX-\hat\mu_11^TX -X^T 1 \hat\mu_1 + \hat\mu_1 1^T1 \hat\mu_1^T \\&=\boxed{X^TX-n_1\hat\mu_1\hat\mu_1^T} \end{aligned}

And similarly $$Y^TY-n_2\hat\mu_2\hat\mu_2^T$$ for the $$y$$-sum. On the other hand

\begin{aligned} \sum_{i=1}^{n_1} (x_i-\hat\mu_0)(x_i-\hat\mu_0)^T &= (X-1\hat\mu_0^T)^T(X-1\hat\mu_0^T) \\&= X^TX-\hat\mu_01^TX -X^T 1 \hat\mu_0 + \hat\mu_0 1^T1 \hat\mu_0^T \\&=\boxed{X^TX - n_1\hat\mu_0\hat\mu_1^T - n_1\hat\mu_1\hat\mu_0 + n_1\hat\mu_0\hat\mu_0^T} \end{aligned}

And similarly $$Y^TY - n_2\hat\mu_0\hat\mu_2^T - n_2\hat\mu_2\hat\mu_0 + n_2\hat\mu_0\hat\mu_0^T$$ for the $$y$$-sum. Adding both together, and using $$\hat \mu_0 = \frac{n_1 \hat\mu_1 + n_2\hat\mu_2}{n_1 + n_2}$$ we find

\begin{aligned} &X^TX - n_1\hat\mu_0\hat\mu_1^T - n_1\hat\mu_1\hat\mu_0 + n_1\hat\mu_0\hat\mu_0^T \\&+Y^TY - n_2\hat\mu_0\hat\mu_2^T - n_2\hat\mu_2\hat\mu_0 + n_2\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-\hat\mu_0(n_1\mu_1^T + n_2\mu_2^T) - (n_1\mu_1 + n_2\mu_2)\hat\mu_0^T +(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-(n_1+n_2)\hat\mu_0\hat\mu_0^T - (n_1+n_2)\hat\mu_0\hat\mu_0^T +(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=X^TX+Y^TY-(n_1+n_2)\hat\mu_0\hat\mu_0^T \\&=\boxed{X^TX+Y^TY-\frac{(n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T}{n_1+n_2}} \end{aligned}

Thus, our goal of showing $$\hat\Sigma_0 = \hat\Sigma +uv^T$$ is equivalent to (after cancelling $$X^TX$$ and $$Y^TY$$ terms)

$$-\frac{(n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T}{(n_1+n_2)^2} =\frac{-n_1\hat\mu_1\hat\mu_1^T-n_2\hat\mu_2\hat\mu_2^T}{n_1+n_2} + \frac{n_1n_2(\hat \mu_1 -\hat\mu_2)(\hat \mu_1 -\hat\mu_2)^T}{(n_1+n_2)^2}$$

Which is equivalent to

$$(n_1\hat\mu_1+n_2\hat\mu_2)(n_1\hat\mu_1+n_2\hat\mu_2)^T + n_1n_2(\hat \mu_1 -\hat\mu_2)(\hat \mu_1 -\hat\mu_2)^T =(n_1+n_2)(n_1\hat\mu_1\hat\mu_1^T+n_2\hat\mu_2\hat\mu_2^T)$$

which is a true statement. q.e.d

Final Remark: Of course, the Matrix Determinant Lemma would also offer a strategy of figuring out the equation in the first place: If we wanted to find the relationship bewtween $$\det(\hat\Sigma_0)$$ and $$\det(\hat\Sigma)$$, it would suggest to express $$\hat\Sigma_0 = \hat\Sigma + UV^T$$. Since both $$\hat\Sigma_0$$ and $$\hat\Sigma$$ are symmetric, we are in fact guaranteed $$\hat\Sigma_0 = \hat\Sigma + UU^T$$ for some $$(p\times k)$$ matrix $$U$$ with $$k\le p$$ (generally we would choose the $$U$$ matrix with $$k$$ minimal)

• Very nice @Hyperplane, very very nice. I learned a few things today, thank you. – A rural reader Jan 31 at 4:18
• Thank you very much for the question. Would you mind taking a look at this question? math.stackexchange.com/questions/3718725/… – user634512 Jan 31 at 11:21
• It is a nice solution. (+1) – River Li Feb 4 at 1:37

Python simulation showing that the right-hand side must contain $$\hat\Sigma^{-1}$$ (equation in comments), not $$\Sigma^{-1}$$ (your original equation)

import numpy as np
from numpy.linalg import det, inv
from scipy.stats import invwishart, multivariate_normal as normal

n1, n2, p = 10, 12, 5
μ1, μ2 = normal.rvs(size=(2,p))
Σ  = invwishart(df=p+1, scale=np.eye(p)).rvs()
X  = normal(mean=μ1, cov=Σ).rvs(n1)
Y  = normal(mean=μ2, cov=Σ).rvs(n2)

μ0hat = (X.sum(axis=0) + Y.sum(axis=0))/(n1+n2)
μ1hat = X.mean(axis=0)
μ2hat = Y.mean(axis=0)

Σ0hat  = ((X-μ0hat).T@(X-μ0hat) + (Y-μ0hat).T@(Y-μ0hat))/(n1+n2)
Σhat= ((X-μ1hat).T@(X-μ1hat) + (Y-μ2hat).T@(Y-μ2hat))/(n1+n2)
LHS = det(Σ0hat)/det(Σhat)
RHS = 1 +  ((n1*n2)/(n1+n2)**2) * (μ1hat-μ2hat)@inv(Σ)@(μ1hat -μ2hat)
print(LHS-RHS)
corrected_RHS = 1 +  ((n1*n2)/(n1+n2)**2) * (μ1hat-μ2hat)@inv(Σhat)@(μ1hat -μ2hat)
print(LHS-corrected_RHS)