Positive definite matrices? I read that if I add a positive-definite (PD) matrix to a positive-semidefinite (PSD) matrix, then their sum is non-singular (or even PSD). But does that mean that positive-semidefinite matrices are always non-singular? That sounds wrong to me.
Also, is it true that $A^T A$ is always PD or PSD?
 A: Recall that an $n \times n$ matrix $A$ is positive-definite if and only if $A = B^T B$ for some $m \times n$ matrix $B$ whose columns are linearly independent. We have therefore that $A^T = (B^T B)^T = B^T (B^T)^T = B^T B = A,$ hence $A$ is symmetric, from which it follows that $A^T A = A^2.$ By the Spectral Theorem, we have that $A$ is orthogonally diagonalizable. Particularly, there exists an invertible matrix $P$ such that $A = PDP^{-1},$ where $D$ is the diagonal matrix whose nonzero entries are the eigenvalues of $A.$ But of course, we have that $$A^2 = (PDP^{-1})^2 = (PDP^{-1})(PDP^{-1}) = PDP^{-1}PDP^{-1} = PDIDP^{-1} = PD^2P^{-1},$$ hence the eigenvalues of $A^2$ are precisely the squares of the eigenvalues of $A.$
Consequently, if $A$ is a positive-definite $n \times n$ matrix, then $A^T A$ is positive-definite. Certainly, we could repeat this entire argument with the relaxed assumption that $A$ is only positive-semidefinite, in which case we would find that $A^T A$ is positive-semidefinite. Of course, a positive-semidefinite matrix could be singular, as it is by definition possible that $0$ is an eigenvalue.
