# Questions tagged [principal-component-analysis]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

144 questions
Filter by
Sorted by
Tagged with
38 views

### Check if 2 point clouds are the same up to coordinate flips and rotation

I want to check if 2 point clouds in N dimensions are the same up to rotations about the origin and coordinate swaps. I define a point cloud as a finite collection of points, already centered at the ...
1 vote
86 views

### Prove that PCA decomposition captures all information in a factor model

Assume a data matrix $X \in \mathbb{R}^{N \times p_X}$. Let it have some exact lower dimensional factor representation $X = A F$, where $F \in \mathbb{R}^{N \times p_F}$ and $p_F < p_X$. Let the ...
• 1,671
1 vote
43 views

### Proving Optimal Loss Analogous to PCA

Suppose we have a data distribution on $x ∈ R^d$. Suppose $E_x[x] = 0$, and let $Σ = E_x[xx^T]$ be the covariance of x. Let $Σ = USU^T$ be the spectral decomposition of Σ, with U orthonormal and S ...
• 11
67 views

### minimizing $| x x^T - A |^2$ for a covariance matrix $A$

For a given covariance matrix $A$ (so $A$ is symmetrical and positive semidefinite), I want to find a vector $x$ such that $| x x^T - A |^2$ is minimized. Here $|M|^2$ just means the sum of squares ...
• 1,634
1 vote
63 views

• 360
118 views

### Proof that PCA is equivalent to MDS when using Euclidean distances

As I was watching a video explaining how MDS works, the narrator mentioned that PCA is equivalent to MDS when Euclidean distances are used. I got confused as to how that's the case. My guess is that ...
• 121
1 vote
25 views

### Relation of principal component analysis between a matrix and its transopose

With a matrix $X_{n\times p}$ ($n>p$), we perform a principal component analysis: $T_{n\times p}=X_{n\times p}W_{p\times p}$ where $W$ is the loadings matrix while $T$ is the scores matrix for $X$. ...
• 153
1 vote
81 views

### Can SVM be special case of PCA?

Let $X$ and $Y$ two linearly separable finite subsets of a $K$-dimensional real vector space $V$ with orthonormal basis $A = \{a_1,\ldots, a_K\}$. The covariance matrix $\Sigma_A$ of the set $X \cup Y$...
61 views

28 views

### Find line with respect to which the moment of inertia is minimized

Consider a function $F(x,y,z): \mathbb{R}^3 \mapsto \mathbb{R}^+$ (i.e., $F(x,y,z) > 0 ~~\forall~ x,y,z$) and consider a set of points in the (3D) space, $\{p_1, p_2, \cdots , p_N\}$. The problem ...
• 144
150 views

### Eigenvectors, Singular Vectors, and Excel

First time asker here. First off, I know I should be doing this in R or Python. I will. For now I'm reading a textbook, using simple examples and Excel to try to learn the concepts of linear algebra. ...
• 13
47 views

### How to find two small matrices $M_1$ and $M_2$ such that $M_1 M_2 A \approx M A$?

If we have a matrix $M$ and we want to find its least squares approximation as the product of two smaller (as in less rows or columns) matrices $M_1M_2$ of a given size, we can simply run SVD and pick ...
1 vote
120 views

### Clarifying the constraints used in deriving the Principal Components of PCA

In studying principal components analysis, I am confused by one point. For a set of $N$ (zero-centered) data points of dimension $m$, projected to a dimension $k < m$, we want a set of vectors of ...
• 3,848
1 vote
107 views

### Understanding the solution to the varimax rotation problem

I'd like to preface this post by saying that this is my first post on stack exchange, so if there is anything to improve, be it redaction or just the structuring of posts, I'm more than willing to ...
1 vote
30 views

### Can you use EVD, SVD, PCA to solve least squares? (Intuitive Understanding)

I am trying to fully understand/demystify EVD, SVD and PCA. Am I right to assume all of these are tools/methods to solve least squares (not only but for CG)? If I am not wrong, even there are multiple ...
• 111
29 views

### strategies for looking at the phase space of a system with 6 dimensions

I have a system of odes where the state vector has 6 elements. The system is a population biology model, where I am tracking the evolution of some competing species over time. Now I was trying to ...
• 2,531
1 vote
472 views

### Computation of exponential and logarithmic maps on Riemann manifolds

In my computational problem, I have a Riemann submanifold $S^{1000}$ embedded in $\mathbb{R}^{300000}$. I can numerically compute the induced metric tensor and the Jacobian. I have no analytical ...
• 11
1 vote
302 views

### Proof of the variance of one-dimensional projections

In Bishop's Pattern Recognition And Machine Learning book, Chapter 12, Suppose $X$ is an uncentered data matrix and $\bar{x}=\frac{1}{m}\sum_ix_i$ is the sample mean of the columns of $X$. For the ...
• 303
188 views

### Where in PCA does the non-uniqueness of eigenvectors come from?

I tried comparing sklearn.decomposition.KernelPCA with a linear kernel to sklearn.decomposition.PCA on the same data set and got different eigenvectors. My understanding is that these should be ...
• 1,876
118 views

### Proof of "The sum of squared distances from the points to the line is a minimum"

I'm reading "Introduction to linear algebra" of Gillbert Strang. PCA by SVD section. Text says that the sum of squared distances from the points to the line is a minimum and author is trying ...
• 51
104 views

### quasi-PCA reconstruction of the matrix by orthogonal basis

let's say I have a "data" matrix $X$ of $N$ rows and $p$ cols with $N \gg p$. Now PCA with $L$ components can be formulated as $$X_L = argmin_{Y:rank(Y) = L} ||X- Y||^2_F$$, where Y is ...
241 views

### How to compute principal components for a curvature found given XYZ points?

I have a certain XYZ set of points that make up an object. I chose a random point and make the nearest radius analysis and find the neighbors. From these neighbors, I get the green pointcloud curve ...
1k views

### projection of the data along the 1st k principal components

I'm a final year maths undergrad doing a course in multivariate data analysis, but I'm really struggling with the linear algebra. In particular the “projection of the data along the 1st k principal ...
• 39
1 vote
26 views

### PCA factor $\frac{1}{n-1}$

I've seen multiple PCA derivations where the $\frac{1}{n}$ (for variance) or $\frac{1}{n-1}$ (for sample variance) is just omitted, e.g. here. I see that they are proportional to each other but is ...
• 333
28 views

### How to learn data points by minimizing a loss function given their pairwise distance matrix?

Suppose $x_i \in\mathbb{R}^2$ for $i=1,2,...9$ are unknown. I'm given the pair-wise distance matrix between these points $D$ which is a $9*9$ symmetric matrix. I want to learn these data points by ...
• 2,533
1 vote
999 views

### What does singular value decomposition of covariance matrix represent?

I am reading the paper "Understanding dimensional collapse in contrastive self-supervised learning." The authors identified a dimensional collapse phenomenon: i.e. some dimension of ...
• 11
1 vote