How to take the inverse of the matrix $X^{T}X$, when it isn't invertible? If I have a matrix $X$ and I am trying to compute $(X^{T}X)^{-1}$, which is the inverse of $X^{T}X$. However, each time I try to do it in some computing package like R, I get that $X^{T}X$ is singular. The matrix $X$ looks like:
$$X = \begin{bmatrix}
 -1 & 0 & -1 & -1 & 0 & -1 \\ 
   -1 & 0 & -1 & 1 & 1 & 0 \\ 
   -1 & 0 & 1 & -1 & 0 & 1 \\ 
   -1 & 0 & 1 & 1 & -1 & 0 \\ 
  1 & -1 & 0 & -1 & 0 & 1 \\ 
   1 & -1 & 0 & 1 & -1 & 0 \\ 
   1 & 1 & 0 & -1 & 0 & -1 \\ 
   1 & 1 & 0 & 1 & 1 & 0 \\ 
 \end{bmatrix}
$$
This question is from a paper I read, where they computed $(X^{T}X)^{-1}$ to be:
$$(X^{T}X)^{-1} = \begin{bmatrix}
 0.125 & 0 & 0 & 0 & 0 & 0 \\ 
   0 & 0.125 & 0 & 0 & 0 & 0 \\ 
   0 & 0 & 0.375 & -0.125 & 0 & -0.25 \\ 
   0 & 0 & -0.125 & 0.375 & 0 & 0.25 \\ 
  0 & 0 & 0 & 0 & 0.125 & 0 \\ 
   0 & 0 & -0.25 & 0.25 & 0 & 0.5 \\ 
 \end{bmatrix}
$$
Does anyone have any idea how to get such a inverse matrix? Thanks.
 A: They probably computed the pseudo-inverse.  Using the Python numpy package you can get it from the pinv function:

In [14]: X
Out[14]:
array([[-1,  0, -1, -1,  0, -1],
       [-1,  0, -1,  1,  1,  0],
       [-1,  0,  1, -1,  0,  1],
       [-1,  0,  1,  1, -1,  0],
       [ 1, -1,  0, -1,  0,  1],
       [ 1, -1,  0,  1, -1,  0],
       [ 1,  1,  0, -1,  0, -1],
       [ 1,  1,  0,  1,  1,  0]])

In [15]: np.linalg.pinv(X.T.dot(X)).round(5)
Out[15]:
array([[ 0.125  ,  0.     ,  0.     ,  0.     ,  0.     ,  0.     ],
       [ 0.     ,  0.15625,  0.09375,  0.     ,  0.03125, -0.03125],
       [ 0.     ,  0.09375,  0.15625, -0.     , -0.03125,  0.03125],
       [ 0.     , -0.     ,  0.     ,  0.125  ,  0.     ,  0.     ],
       [ 0.     ,  0.03125, -0.03125, -0.     ,  0.15625,  0.09375],
       [ 0.     , -0.03125,  0.03125, -0.     ,  0.09375,  0.15625]])

In R, ginv should produce a similar result.
A: When $X$ is not full rank, $X^TX$ is not invertible, and in such a case, you end up having to use a generalized inverse of $X^TX$, denoted $(X^TX)^{-}$. There are many such solutions $(X^{T}X)^{-}$. For statistical purposes, most people just care about the projection matrix $X(X^{T}X)^{-}X^{T}$ which is the same matrix regardless of the generalized inverse that is chosen.
In practice, one would just use a program for finding a generalized inverse (such as R's ginv); however, two methods that I know of (for a square matrix) consists of zeroing-out the $i$th row and the $i$th column, and finding the regular inverse of the resulting matrix with the $i$th row and $i$th column removed (where $i$ is from $1$ to the number of rows/columns of your matrix). The other method I don't recall at the moment, but it is equivalent to what it is known as a "sum-to-zero" constraint on experimental design parameters.
