Many nonlinear manifold learning methods used for dimensional reduction (isomap, diffusion maps, local preserving projections, etc) solve the symmetric eigenvalue problem and then use the eigenvectors whose eigenvalues are the two "most trivial" as the reduced dimensions. Would these always be the smallest (non-zero) eigenvalues and if so, isn't the noise and outlier residual values usually in such eigenvectors? Why not the two largest eigenvalues? Given the trivial viewpoint, what is being done to send or pack information into the eigenvectors with the smallest eigenvalues?

Looking at the fundamental definition of the eigenvalue, that is, an eigenvalue represents the variance of elements in its associated eigenvector, there must be a technique being exploited which nevertheless packs informativeness about class structure into the two eigenvectors while keeping the variance down. Hence, you are taking the eigenvectors, which typically only carry large outlier (extreme) values, and packing information into them that's representative of the original cluster structure at higher dimensions. Thus, "junk" eigenvectors are assigned to carry all the information when done.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.