3
$\begingroup$

I am taking a Phd class in econometrics, and the following is used constantly, for $X$ a $n\times k$ matrix, $n \neq k$: $$(X'X)^{-1} = ((X'X)^{-1})'$$ with "$'$" standing for transpose. Having a rather weak background in linear algebra, I cannot understand why this is true.

For example: ${\underset{k\times1}{\underbrace{\left(\underset{k\times k}{\underbrace{\left(X'X\right)^{-1}}}\underset{k\times1}{\underbrace{X'u}}\right)}}\underset{1\times k}{\underbrace{\left(\underset{k\times k}{\underbrace{\left(X'X\right)^{-1}}}\underset{k\times1}{\underbrace{X'u}}\right)'}}}={\underset{k\times k}{\underbrace{\left(X'X\right)^{-1}}}\underset{k\times n}{\underbrace{X'}}\underset{n\times n}{\underbrace{uu'}}\underset{n\times k}{\underbrace{X}}\underset{k\times k}{\underbrace{\left(X'X\right)^{-1}}}}$

To my understanding the far right expression on the RHS should be different.

$\endgroup$
3
$\begingroup$

Basically, what this says is that $(X^TX)^{-1}$ is symmetric. There are two results about matrices at work here.

First, the inverse of a symmetric matrix (if it exists) is itself symmetric, i.e., $(A^{-1})^T=A^{-1}$. See this question for a proof.

Second, for any matrix $X$, $X^TX$ is symmetric. To see this, consider a typical element in the product: $(X^TX)_{ij}=\sum_k(X^T)_{ik}X_{kj}=\sum_kX_{ki}X_{kj}$, but this is just the dot product of the $i$th and $j$th columns of $X$. Since the dot product is symmetric, $(X^TX)_{ij}=(X^TX)_{ji}$. Combining these two results gives you $((X^TX)^{-1})^T=(X^TX)^{-1}$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Note that you can show directly that $(X^TX)^T=X^TX$ without diving down into individual elements, but I wanted to highlight the fact that this matrix contains all of the possible dot products of a set of vectors. $\endgroup$ – amd Dec 2 '15 at 21:58
4
$\begingroup$

First: $(X'X)'=X'(X')'=X'X$. Then:

$(X'X)^{-1}(X'X) = 1 = 1' = \big((X'X)^{-1}(X'X)\big)'=(X'X)'\big((X'X)^{-1}\big)'=(X'X)\big((X'X)^{-1}\big)'$

So $1=(X'X)\big((X'X)^{-1}\big)'$ which means $\big((X'X)^{-1}\big)' = (X'X)^{-1}$.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

First, note that $(X'X)'=X'(X'')=X'X$. Now suppose $A$ is an invertible matrix such that $A'=A$, with inverse $B$. I claim that $B=B'$; taking $A=X'X$ will then give your equation. Indeed, note that $AB=BA=I$, so taking transposes, $B'A'=A'B'=I'=I$, so $B'$ is an inverse of $A'$. Since $A'=A$, this means $B'=A^{-1}=B$.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.