I've pretty much been trying to learn PCA all day, and I feel like I'm so close. Here are my two main questions right now. Hopefully, someone on this thread can help.
First of all, I THINK I understand the computational process...but if someone can go through it as an answer just in case, I'd still really appreciate it!
So, we start with a dataset, we reduce its dimensions by giving it new features that are each a linear combination of the original features of the dataset, and only keep the features with maximum variance.
These new features of our reduced dataset are eigenvectors of the covariance matrix of our original dataset. For some reason, making the new features the eigenvectors of the original covariance matrix does two things:
These new features will have a much larger variance than any of the features in the original dataset had. I don’t see why…
These new features will all have zero covariance with one another…I also don’t see why. Well, I kind of do when I follow the math, but it would be helpful to hear it in words…
Also, I know that this is a data science question, but I'd like to hear it explained by mathematicians with a geometrical appreciation of linear algebra...idk, I just like understanding things geometrically better, and I feel like data science often doesn't address the geometric view of linear algebra...I find it to be the most beautiful.