variance matrix of any unbiased estimator - var(UMVUE) is non-negative definite proof consider a parameter vector $\vec{\theta}=(\theta_1,\ldots,\theta_k)^T$. let $\vec{T}=(T_1,\ldots,T_k)^T$ such that each $T_i$ is the UMVUE of $\theta_i$ (not necessarily independent). Show that for every unbiased estimator $\vec{S}=(S_1,\ldots,S_k)^T$ of $\vec{\theta}$, $V(\vec{S})-V(\vec{T})$ is non-negative definite.
I see a similar proof here http://house.cc.gt.atl.ga.us/~lebanon/pub/chapter10.pdf on page 43 for cramer-rao lower bound, but UMVUE does not necessarily attain CRLB?
appreciate any comments. thank you!
 A: $\newcommand{\var}{\operatorname{var}}\newcommand{\cov}{\operatorname{cov}}\newcommand{\E}{\operatorname{E}}$I'll follow the convention
$$
\var(\vec S) = \E\Big((\vec S-\E\vec S)(\vec S-\E\vec S)^T\Big) \in\mathbb R^{k\times k};
$$
I'm assuming for now that this is what you meant by $V(\vec S)$.  Likewise for $\vec R=(R_1,\ldots,R_\ell)$,
$$
\cov(\vec R,\vec S)=\E\Big( (\vec R-\E\vec R)(\vec S-\E\vec S)^T \Big) \in\mathbb R^{\ell\times k}.
$$
Then
$$
\var(\vec S) = \var((\vec S-\vec T)+\vec T) = \var(\vec S-\vec T)+\cov(\vec S-\vec T,\vec T)+\cov(\vec T,\vec S-\vec T)+ \var(\vec T).
$$
If we can show that the two covariances are $0$ then we have
$$
\var(\vec S-\vec T)=\var(\vec S)-\var(\vec T)
$$
and it follows that that expression on the right is non-negative definite.
Thus it is enough to show that if $\vec E$ is an unbiased estimator of $0$ then $\cov(\vec E,\vec T)=0$.  Suppose this covariance is not $0$.  Then consider
$$
T_k - \frac{2\cov(T_k,E_k)}{\var(E_k)}E_k.
$$
The problem is then to show that this has a smaller variance than does $T_k$, thereby showing that it is a better estimator than $T_k$.  And that is a contradiction.
