19
$\begingroup$

$\newcommand{\Var}{\operatorname{Var}}$In this video is claimed that if the equation of errors in OLS is given by: $$u=y - X\beta$$ Then in the presence of heteroscedasticity the variance of $u$, will not be constant, $\sigma^2 \times I$, where $I$ is an identity matrix, but: $$\Var(u\mid X)=\sigma^2\Omega$$ In order to account for the heteroskedasticity, we can estimate the transform system, such that $P$ is a transformation matrix. $$Py=PX\beta-Pu$$
Where "the variance of a constant matrix $P$ times a random vector $u$" is: $$\Var(Pu\mid X)=P\Var(u\mid X)P'=P(\sigma^2\Omega)P'$$ Can somebody explain me the proof for that?

$\endgroup$
6
  • $\begingroup$ You are right, I don't understand why the variance of a constant matrix P, times a random vector u, is Var(Pu)=PuP' why? $\endgroup$
    – Mario GS
    Jul 20, 2017 at 15:59
  • 3
    $\begingroup$ I believe you do not have any problems with first three equations but variance of a linear transform. $$Var(P u) = E[(P(u-u_{mu}))^2] $$ $$Var(P u) = E[(P(u-u_{mu}))(P(u-u_{mu}))^{H}]= E[P(u-u_{mu})(u-u_{mu})^{H} P^{H}]=PE[(u-u_{mu})(u-u_{mu})^{H}] P^{H}=PVar(u)P^{H}$$ $\endgroup$
    – keoxkeox
    Jul 20, 2017 at 16:03
  • $\begingroup$ You can replace $u$ with $u$|$X$ as they both are random variables and I used $u$ as a dummy variable above. So Var(Pu)=PVar(u)P' with your notation $\endgroup$
    – keoxkeox
    Jul 20, 2017 at 16:05
  • $\begingroup$ Great!, in this case, 'H', is the transpose, right? $\endgroup$
    – Mario GS
    Jul 20, 2017 at 16:12
  • $\begingroup$ Yes transpose for real signals/vectors and Hermitian for complex ones. and $u_mu$ is the expectation of $u$ $\endgroup$
    – keoxkeox
    Jul 20, 2017 at 16:13

1 Answer 1

32
$\begingroup$

$$ \operatorname{var}(AX) = A\Big( \operatorname{var}(X) \Big) A^T. $$

  • $X\in\mathbb R^{\ell\times1}$ is a random column vector,
  • $\operatorname{var}(X) = \operatorname{E}((X-\mu)(X-\mu)^T)$, where $\mu=\operatorname{E}(X),$ is an $\ell\times\ell$ constant (i.e. non-random) matrix,
  • $A\in\mathbb R^{k\times\ell}$ is a constant matrix,
  • and so $\operatorname{var}(AX)\in\mathbb R^{k\times k}$ is a constant matrix.

The proof is this: \begin{align} & \operatorname{var}(AX) \\[10pt] = {} & \operatorname{E}\Big((A(X-\mu))(A(X-\mu))^T\Big) \\[10pt] = {} & \operatorname{E}\Big(A(X-\mu)(X-\mu))^T A^T\Big) \\[10pt] = {} & A \operatorname{E}\Big((X-\mu)(X-\mu))^T \Big) A^T \\[10pt] = {} & A \Big( \operatorname{var}(X) \Big) A^T. \end{align}

$\endgroup$
1
  • $\begingroup$ This is very similar to what keoxkeox posted before. TX $\endgroup$
    – Mario GS
    Jul 21, 2017 at 10:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .