What is the significance of reversing the polarity of the negative eigenvalues of a symmetric matrix? Consider a full rank $n\times n$ symmetric matrix $A$ (coming from a set of physical measurements). I do an eigendecomposition of this matrix as
$$A = E V E^T$$
Most of the eigenvalues are positive, while a few are negative but with much smaller magnitude compared to the maximum eigenvalue. I want to convert this matrix into a positive definite one but with minimal damage, for the purpose of my experiments. So I naively invert the polarity of the negative eigenvalues and construct a PSD matrix
$$B = E |V| E'$$
Later, I realized there is a whole lot of research and algorithms available to find the nearest PSD matrix to a given symmetric matrix and what I am doing is not the right way. So I got hold of a few of those algorithms (with code) and tried out. These algorithms give nearest PSD matrix $C$ to $A$ than $B$ in terms of $2$-norm and Frobenius norm. Surprising part is when I use them in my experiments, I get far better results with $B$ than any of those algorithms. My understanding is that those algorithms usually end up in a low rank matrix (semi definite) and they do $C+\lambda I$ to make it make it a PSD where we get to choose $\lambda$.
I am not able to understand fully why $B$ is giving good results than $C$ and what is the significance of $B$. I'd like to understand this as i want better solution than naively going for $B$ albeit it gives results.
 A: Presumably, the "true" matrix that you're measuring really is only positive semidefinite with a large kernel.  Since you're doing physical measurements, the chance of getting any number exactly is nil, so all the zero eigenvalues are coming out as either tiny positive or tiny negative numbers.
If you flip the negative eigenvalues to positive ones, you're generating a positive definite matrix which behaves somewhat similarly to the "true" positive-semidefinite matrix.  I don't know what you're doing with these subsequently, but you may be feeding them through code that only works properly for positive definite matrices or something.
Try this: instead of flipping the signs, replace them with randomly-generated positive numbers of roughly the same order of magnitude and see what happens.  If the results are basically identical to the sign-flipping, then the problems you're having with the PSD matrix are probably some kind of numerical shenanigans.  If the flipped-signs version is behaving noticeably better (whatever that means in this context), then there might be something really interesting going on.
Alternatively: How sure are you that the software you're using to find the nearest PSD matrix is working correctly?
A: There can be no definite answer because it has been left unspecified what
good results are and what the origin of the undesired negative eigenvalues is.
Anyway, the following might have to do with it:


*

*Adding $\lambda I$ to $C$ also distorts the originally positive eigenvalues,
whereas $B$ leaves those unchanged.

*$B$ is auto-adaptive: The errors in $B$ automatically grow and shrink with
the absolute values of the negative eigenvalues. In particular,
if there are no negative eigenvalues, $B$ remains identical to $A$.
In contrast, when adding $\lambda I$, care must be taken to ensure that
$\lambda$ is large enough in all cases, and this adds unnecessary deviation
from $A$, particularly when negative eigenvalues happen to be much smaller
in magnitude than $\lambda$.

*That said, your way of approximating $A$ with $B$ is not robust:
If the diagonalization happens to find a zero eigenvalue, $B$ will still have
that, and will not be positive definite, only positive semidefinite.
It seems however that you have actively tried to prevent such cases.
This gets us to the next point.

*You mentioned that you obtain $A$ by removing redundant rows/columns from a
matrix $M$.
To recognize redundant rows, the row reduction algorithm repeatedly gets
to a point where it needs to find out whether some number is approximately
zero. Matlab allows specification of tolerance values there. If you set the
tolerance too low (e.g. smaller than typical experimental error margins),
then too few rows will be removed, and you get the effect that Daniel
McLaury has described: small eigenvalues with random signs in place of zero
eigenvalues. Instead of processing such artefacts, better avoid them by
using a larger tolerance for the row reduction.

*You might also want to consider using methods that robustly work with
semidefinite matrices. This might save you from having to reduce $M$
in the first place, so you get a much more straightforward analysis.
SVD and pseudo-inverses come to mind. But I am only guessing here.

A: Given an invertible symmetric matrix $A \in \mathbb{R}^{n \times n}$, an eigendecomposition of $A$ is
$$A = Q \Lambda Q^T = \begin{bmatrix} Q_1 & Q_2\end{bmatrix} \begin{bmatrix} \Lambda_1 & O\\ O & \Lambda_2\end{bmatrix} \begin{bmatrix} Q_1^T\\ Q_2^T\end{bmatrix}$$
where the columns of orthogonal matrix $Q$ are the eigenvectors of $A$ and the entries on the main diagonal of diagonal matrix $\Lambda$ are the eigenvalues of $A$.
We build a new matrix $B \in \mathbb{R}^{n \times n}$ that has the same eigenvectors as $A$. However, the eigenvalues in the block $\Lambda_2$ have their signs changed, i.e.,
$$B = \begin{bmatrix} Q_1 & Q_2\end{bmatrix} \begin{bmatrix} \Lambda_1 & O\\ O & -\Lambda_2\end{bmatrix} \begin{bmatrix} Q_1^T\\ Q_2^T\end{bmatrix}$$
We write
$$A = Q_1 \Lambda_1 Q_1^T + Q_2 \Lambda_2 Q_2^T$$
$$B = Q_1 \Lambda_1 Q_1^T - Q_2 \Lambda_2 Q_2^T$$
Hence,
$$B = A - 2 \, Q_2 \Lambda_2 Q_2^T = \underbrace{(I_n - 2 \, Q_2 \Lambda_2 Q_2^T A^{-1})}_{=:R} \, A = R A$$
We now study matrix $R$,
$$\begin{array}{rl} R &= I_n - 2 \, Q_2 \Lambda_2 Q_2^T A^{-1}\\\\ &= I_n - 2 \, Q_2 \Lambda_2 Q_2^T (Q_1 \Lambda_1^{-1} Q_1^T + Q_2 \Lambda_2^{-1} Q_2^T)\\\\ &= I_n - 2 \, Q_2 \Lambda_2 \underbrace{Q_2^T Q_1}_{= O} \Lambda_1^{-1} Q_1^T - 2 \, Q_2 \underbrace{\Lambda_2 Q_2^T Q_2 \Lambda_2^{-1}}_{= I} Q_2^T)\\ &= I_n - 2 \, Q_2 Q_2^T\end{array}$$
which looks very much like a Householder reflection. Note that $Q_2 Q_2^T$ is a projection matrix whose rank and trace are equal to the number of eigenvalues whose signs are changed. Since $R$ is symmetric, it has an eigendecomposition
$$\begin{array}{rl} R &= I_n - 2 \, Q_2 Q_2^T\\\ &= Q Q^T - 2 \, Q_2 Q_2^T\\ &= Q_1 Q_1^T - Q_2 Q_2^T\\ &= \begin{bmatrix} Q_1 & Q_2\end{bmatrix} \begin{bmatrix} I & O\\ O & -I\end{bmatrix} \begin{bmatrix} Q_1^T\\ Q_2^T\end{bmatrix}\end{array}$$
Thus, $R$ is involutory
$$R^2 = I_n \implies R^{-1} = R$$
and, since it is symmetric, it is also orthogonal. Hence, Euclidean distances are preserved
$$\|R x\|_2 = \sqrt{x^T R^T R x} = \sqrt{x^T x} = \|x\|_2$$
The Frobenius norm is also preserved
$$\|R X\|_F = \sqrt{\operatorname{tr} (X^T R^T R X)} = \sqrt{\operatorname{tr} (X^T X)} = \|X\|_F$$
The Frobenius norm of $R$ is
$$\|R\|_F = \sqrt{\operatorname{tr} (R^T R)} = \sqrt{\operatorname{tr} (I_n)} = \sqrt{n}$$
The spectral norm of $R$ is $\|R\|_2 = 1$. The determinant of $R$ is
$$\det(R) = (-1)^{m}$$
where $m \leq n$ is the number of sign reversals.
