I wrote a python script which calculates the Rayleigh quotient with gradient descent line search. This is just the original gradient descent as described by Nocedal et al etc. and I use the Armjio condition to check for sufficient decrease.
The code works fine for normal matrices. I checked the min eigenvalue I get with those calculated from numpy.linalg.eigh(A) and they are all matching closely.
However, if I have a matrix that has many eigenvalues that are 0, but each eigenvalue has a different eigenvector, I have trouble converging to any of these eigenvectors. I get other vectors that get my Rayleigh quotient down to 0, but they do not match what I get from numpy.linalg.eigh(A). (but I can converge to eigenvector that corresponds to the max eigvalue which is not 0).
I need help in how to converge to the eigenvector when my eigenvalue is 0.
However, ultimately, I want to skip all these 0 eigenvalues and get the first eigenvector for the eigenvalue that is NOT 0. If anyone can guide me to such an algorithm that would be fine as well!