The meaning behind $(X^TX)^{-1}$ In linear algebra, we learn that the inverse of a matrix "undoes" the linear transformation.  What exactly is the meaning of the inverse of $(X^TX)^{-1}$? 
$X^TX$ we know as being a square matrix whose diagonal elements are the sums of squares.  So what are we doing when we take the inverse of this?  I have always used this property in my calculations but would like to understand more of the meaning behind it.
 A: When $X$ is a real matrix, the elements of $(X^TX)^{-1}$ also provide a measure of the extent of linear dependence among the columns of $X$.
If $X^TX$ is invertible then the columns of $X$ have to be independent, but sometimes the the columns are "almost" dependent in a sense which will be made clear below.
Denote the $i$th column of $X$ by $x_i$ and let let $\hat{x_i}$ denote the projection of $x_i$ on  space spanned by $\{x_j : j \neq i \}$. Call $\epsilon_i = x_i - \hat{x_i}.$ Not that if any $\|\epsilon_i\|$ is "small", it suggests strong linear dependence among the columns of $X$
One can prove the $ij$th element of $(X^TX)^{-1}$ is $\dfrac{\epsilon_i^T\epsilon_j}{\|\epsilon_i\|^2\|\epsilon_j\|^2}.$
In particular the ith diagonal element of $(X^TX)^{-1}$ is $\dfrac{1}{\|\epsilon_i\|^2}$. So if the $i$th column of $X$ is almost a linear combination of other columns, it will be indicated by a very large value at the $i$th diagonal element of $(X^TX)^{-1}$.
---Added later---
We can prove the expression for the elements of the inverse as follows.
Assume we have $p$ independent columns $x_1,\dots,x_p$. Let $\hat{x_i}$ be the projection of $x_i$ on the space spanned by $\{x_j : j \neq i\}$ and let $e_i = x_i - \hat{x_i}$
By the definition of orthogonal projection $e_i$ is orthogonal to any vector in the span of $\{x_j : j \neq i\}$.
So $e_i^T x_j = 0, j \neq i$.
Since $\hat{x_i} = x_i - e_i$ is in the space spanned by $\{x_j : j \neq i\}$. So $e_i^T (x_i - e_i) = 0$, i.e., $e_i^T x_i = \|e_i\|^2$.
$e_1,\dots,e_p$ are independent because $\sum_{i=1}^p a_i e_i = 0$ implies, for any $j$, $(\sum_{i=1}^p a_i e_i )^T x_j = 0$, i.e., $a_j e_j^T x_j =  a_j \|e_j\|^2 = 0$ so $a_j = 0.$
Since $\text{span}\{e_1,\dots,e_p\} \subset \text{span}\{x_1,\dots,x_p\}$ the independence of $e_i$s implies $\{e_i\}$ and $\{x_i\}$ are different bases of the same space.
There exist $p \times p$ matrices $A=(a_{ij})$ and $B=(b_{ij})$ such that
$$ x_i = \sum_{k=1}^p a_{ik} e_k $$ and $$ e_i = \sum_{k=1}^p b_{ik} x_k$$
By the change of basis formula, $A$ and $B$ must be inverses of each other.
Note $x_i^T x_j =  ( \sum_{k=1}^p a_{ik} e_k )^T x_j = a_{ij} e_j^T x_j = a_{ij} \|e_j\|^2$.
So $a_{ij} = \dfrac{x_i^T x_j}{\|e_j\|^2}$, for all $i,j$ i.e., $A = (X^T X) \begin{pmatrix} \dfrac{1}{\|e_1\|^2} & 0 & 0 & \dots & 0  \\ 0 & \dfrac{1}{\|e_2\|^2} & 0 & \dots &  0 \\ \dots & \dots & \dots & \dots \\ 0 
 & 0 & \dots & 0 & \dfrac{1}{\|e_p\|^2}\end{pmatrix}$.
Similarly, we can prove $b_{ij} = \dfrac{e_i^T e_j}{\|e_j\|^2}$.
So, $B = (E^T E) \begin{pmatrix} \dfrac{1}{\|e_1\|^2} & 0 & 0 & \dots & 0  \\ 0 & \dfrac{1}{\|e_2\|^2} & 0 & \dots &  0 \\ \dots & \dots & \dots & \dots \\ 0 
 & 0 & \dots & 0 & \dfrac{1}{\|e_p\|^2}\end{pmatrix}$ where $E$ is the matrix with columns $e_1,\dots,e_p$.
Since $B = A^{-1}$ we have,
$$
(X^T X)^{-1} = \begin{pmatrix} \dfrac{1}{\|e_1\|^2} & 0 & 0 & \dots & 0  \\ 0 & \dfrac{1}{\|e_2\|^2} & 0 & \dots &  0 \\ \dots & \dots & \dots & \dots \\ 0 
 & 0 & \dots & 0 & \dfrac{1}{\|e_p\|^2}\end{pmatrix} E^T E \begin{pmatrix} \dfrac{1}{\|e_1\|^2} & 0 & 0 & \dots & 0  \\ 0 & \dfrac{1}{\|e_2\|^2} & 0 & \dots &  0 \\ \dots & \dots & \dots & \dots \\ 0 
 & 0 & \dots & 0 & \dfrac{1}{\|e_p\|^2}\end{pmatrix}$$ and the result follows.
A: Probably the main intuition you will get from the fact that for OLS model you have
$$
\operatorname{Var}(\hat{\beta}) = \sigma^2_{\epsilon}(X'X)^{-1},
$$ 
namely, you can view $(X'X)^{-1}$ as matrix that in a sense measures the stability of your model. 
