Questions tagged [principal-component-analysis]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

Sparse PCA algorithm clarification

The Sparse PCA algorithm proposed in this paper proposes two ways to calculate sparse principal components, first by using $$\hat \beta = \text{arg min}_\beta ||Z_i - X\beta||^2 + \lambda ||\beta||^2 +...
user avatar
3 votes
1 answer
53 views

How to compute principal components for a curvature found given XYZ points?

I have a certain XYZ set of points that make up an object. I chose a random point and make the nearest radius analysis and find the neighbors. From these neighbors, I get the green pointcloud curve ...
user avatar
0 votes
1 answer
25 views

projection of the data along the 1st k principal components

I'm a final year maths undergrad doing a course in multivariate data analysis, but I'm really struggling with the linear algebra. In particular the “projection of the data along the 1st k principal ...
user avatar
  • 29
1 vote
0 answers
23 views

PCA factor $\frac{1}{n-1}$

I've seen multiple PCA derivations where the $\frac{1}{n}$ (for variance) or $\frac{1}{n-1}$ (for sample variance) is just omitted, e.g. here. I see that they are proportional to each other but is ...
user avatar
2 votes
0 answers
24 views

How to learn data points by minimizing a loss function given their pairwise distance matrix?

Suppose $x_i \in\mathbb{R}^2$ for $i=1,2,...9$ are unknown. I'm given the pair-wise distance matrix between these points $D$ which is a $9*9$ symmetric matrix. I want to learn these data points by ...
user avatar
  • 2,255
0 votes
0 answers
20 views

What does singular value decomposition of covariance matrix represent?

I am reading the paper "Understanding dimensional collapse in contrastive self-supervised learning". Authors identified a dimensional collapse phenomenon, i.e. some dimension of embedding ...
user avatar
  • 1
0 votes
0 answers
10 views

Correlation on principal component analysis[PCA]

I am analysing a dataset on NBA players, using R. I am having some doubts on correlation in pca. I know the eigenvalues represent the variability in each PCA. Each eigenvalue has a eigenvector. I ...
user avatar
1 vote
1 answer
20 views

How to prove some insights regarding a new pca coordinate system

I've a question regarding pca variant. Let $X ∈ \Bbb R^{D×n}$ be a data matrix, $\{u_i\}^{d}i=1$ be the $d$ principal components of $X$, and where $μ ∈ \Bbb R^d$ is the sample mean vector and $1_n ∈ \...
user avatar
0 votes
0 answers
10 views

Getting a better ranking with principal component analysis

I'm reading Alvin C. Rencher's "Methods of Multivariate Analysis". In the introductory part of principal component analysis section, he states: In principal component analysis, we seek to ...
user avatar
  • 319
0 votes
0 answers
23 views

Comparing distributions of binary valued vectors: covariance matrix is enough?

Say we have two discrete distributions, $\vec{y}\sim p_y$ and $\vec{x}\sim p_x$, for both of which, data vectors have binary-valued entries: $\vec{y}\in\{0,1\}^n$ $\vec{x}\in\{0,1\}^n$, where $n$ is ...
user avatar
  • 75
0 votes
0 answers
9 views

When standardizing data, does that imply that the mean and standard deviation will become 0, respectively 1?

As title suggests, I've been wandering about how standardization works when trying to understand how Principal Component Analysis( PCA) works from this tutorial https://medium.com/analytics-vidhya/...
user avatar
0 votes
0 answers
35 views

Distance metric that is insensitive to correlated variables

I'm trying to find a suitable pairwise distance metric where the addition of correlated vectors results in (essentially) no change in the distance. Specifically, consider a set of $k$ vectors each of ...
user avatar
0 votes
0 answers
15 views

Does rotational matrix affect the variance preservation after applying PCA?

Given a matrix X $\in \mathbb{R}^ {M\times N}$ and its projected matrix $\bar{X}$ after applying PCA. The top $D$ principal components preserve 90% of the total variance. Now assume we have a rotation ...
user avatar
  • 103
2 votes
1 answer
59 views

How do you measure the 'explained variance' of arbitrary linear embeddings?

Question Elaboration: When I say 'linear embeddings' I mean a lower-dimensional representation of variables resulting from an arbitrary linear transformation. And when I say 'explained variance' I ...
user avatar
  • 283
1 vote
0 answers
30 views

Kernel PCA similarity matrix analogy

The standard explanation to linear PCA begins with the covariance matrix. That is, for a dataset $D$ of dimension $N \times d$, the covariance matrix is given as $\sum = \frac{D^{T}D}{N}$ where the ...
user avatar
0 votes
0 answers
17 views

Principal Component Analysis - How do these two representations not contradict each other?

enter image description here The representation here basically says each observation in a data set is a sum of products of some Zs and loadings. By Zs, I mean the component scores. Loadings are the ...
user avatar
0 votes
0 answers
9 views

project lengths onto a line

I'm working on a visualization for PCA. $$ \begin{align} \text{pca}_1 = X \pmb v_1 \end{align} $$ Where $X$ is the dataset. $\text{pca}_1$ is the first principal component and $\pmb v_1$ is akin to an ...
user avatar
1 vote
0 answers
31 views

Confused about the relationship between PCA and robust PCA

I recently learned about PCA and robust PCA. I understand that PCA is identifying the principal components by finding the eigenvectors of the covariance matrix (which of course contains information ...
user avatar
  • 21
1 vote
1 answer
44 views

Why is it possible to write the following term when using the summation solution to PCA?

I am going through PCA notes and I am stumped by the last line in the term expansion (see image below). How did we get $-2\sum_{j=1}^N\sum_{i=1}^k\alpha_{ji}x_j^Tv_i$ from the previous two middle ...
user avatar
0 votes
0 answers
19 views

Meaning of covariance of covariance matrix in PCA.

When doing SVD of matrix X, we obtain U,S,V matrices (i.e. svd(X) = U S V'). The U matrix has the interpretation of being the eigenvectors of the covariance matrix X*X' (where X' is the transpose of X)...
user avatar
  • 11
0 votes
1 answer
115 views

compute eigendecomposition to find eigenvalues and eigenvector

So I have matrix A which has (m×d) dimension and X = A.A^T My X has d non-zero distinct eigenvalues and m > d Usually I compute eigendecomposition matrix which has (mxm) dimension to find ...
user avatar
0 votes
0 answers
14 views

What is the difference, if any, between 'Principal Components' and 'Principal Axes' in PCA?

I refer to the following this article, where is it stated that to do PCA on a mean-subtracted matrix X, you can use SVD: X = U S VT. It is then stated that: the columns of U are the principal ...
user avatar
  • 11
0 votes
0 answers
30 views

Concentrating out for PCA estimation

In the Bai (2003) paper "Inferential Theory for Factor Models of Large Dimensions", page 140, for decomposing the data matrix $X$ by PCA, into $X=F \Lambda'+ e$, concentrating out $\Lambda$ ...
user avatar
  • 1
0 votes
0 answers
8 views

Dimensionality reduction on dimensions predictive of another variable

I have a uniform distribution of input values in a multidimensional space. I also have one (or more) output value for each input value. I know that the input values combine in various linear ways to ...
user avatar
3 votes
0 answers
32 views

unit vector after transformation with longer length than the eigenvector with highest eigenvalue

my question is a general one, can a unit vector after transformation have the length longer than length of the eigenvector which highest eigenvalue? is there any proof? purpose is that I want to know ...
user avatar
0 votes
1 answer
47 views

Singular Vector Decomposition and PCA interpretation

The covariance matrix of any data X -> N * D (N samples and D dimension ) would be Cov(X) = E[(X - E[X])(X - E[X])T]. Let's Assume (X - E[X])=Y. Thus Cov(X) = E[YYT]. Now from SVD we know that U ...
user avatar
1 vote
0 answers
30 views

In PCA, how are maximising variance and minimising reconstruction error equivalent?

I understand that, given a normalised data matrix $X$, the PCA algorithm can be formulated as maximisation problem where objective function is: $$ \max_{{\omega}} \omega'X'X\omega \quad\quad \text{ ...
user avatar
1 vote
0 answers
36 views

Help with a PCA procedure.

I'm studying THIS paper which builds an index from a set of observed variables using Principal Component Analysis (PCA). However, the procedure described in Section 3.1 by formulas (6)-(8) confused me....
user avatar
2 votes
1 answer
86 views

What is the correct formula for the covariance matrix?

I am solving about principal component analysis (PCA) and I stumbled upon a place where I need to calculate the covariance matrix, I am seeing varieties of formula. Here are some that I have found: ...
user avatar
  • 21
0 votes
0 answers
21 views

Linear auto-encoders and PCA with unequal input-output

It is a well-known fact that linear auto-encoders are equivalent to PCA, i.e. for the data matrx $X\in {\mathbb R}^{n\times N}$ the task $$ \min_{W\in {\mathbb R}^{n\times k}}||X-WW^TX|| $$ has a ...
user avatar
  • 331
1 vote
0 answers
12 views

How to create separation of "sides"/signs in PCA / eigenvector "directionality of data" analysis?

How to create separation of "sides"/signs in PCA / eigenvector "directionality of data" analysis? Since PCA will only give eigenvectors that show the principal axes, but it does ...
user avatar
  • 6,822
1 vote
0 answers
19 views

Bound for a random matrix and a normal distributed random vector

In the topic of Principal component analysis in high dimensions I have given the following task: Let $X\in\mathbb R^{n\times d}$ and $w\sim\mathcal N(0,\sigma^2I_n)$. Show that for any $\lambda>0$ ...
user avatar
0 votes
0 answers
12 views

Compare Computation Time of PCA Using SVD of Data Matrix VS Eigen Vectors of Covariance Matrix

I know that principal axises can be computed both using SVD of data matrix and eigen vector decomposition of covariance matrix. I also heared that when dimension of data is larger than the number of ...
user avatar
1 vote
0 answers
16 views

Alternative definition of principal component analysis

I'm reading another definition of $PCA$, quite different from what I've been used to see and I'm a little bit confused: given a dataset $X \in \mathbb{R}^{d \times m}$ that we approximate as $X \sim V^...
user avatar
  • 1,702
0 votes
1 answer
52 views

What does SCoTLASS stand for [closed]

To obtain sparse principal components, there's a technique called SCoLASS. Does anyone know what this acronym stands for? I suppose LASS stands for lasso, as it puts a constraint on an l_1-norm, but ...
user avatar
0 votes
0 answers
59 views

Covariance matrix in PCA approach

While reading the brief discussion on the PCA approach in the book Deep Learning (Ian Goodfellow and Yoshua Bengio and Aaron Courville), I could not understand the passage shown in the figure. ...
user avatar
1 vote
0 answers
61 views

PCA-algorithm dependence on number of principal components.

Consider the following idea: we have $(X,Y) \in \mathbb{R}^{n \times (d+k)}$ - initial data. Let's call $\bar{X} = \dfrac{1}{n}\sum_i x_i$ (the same for $\bar{Y}$). We want to predict $y$ by $x$ using ...
user avatar
  • 5,968
0 votes
1 answer
29 views

Show that $\frac{\sum^k_{i=1}\text{Var}(Y_i)}{\sum^p_{i=1}\text{Var}(X_i)}=\frac{\sum^k_{i=1}\lambda_i}{\sum^p_{i=1}\lambda_i}$

Let $X=(X_1,...,X_p)$ be a random vector with $\mathbb{E}(X)=\mu$ and covariance matrix $\text{Cov}(X)=\Sigma$. Define $Y=(Y_1,...,Y_p)^T=\mathbb{E}^TX$, where $E$ is a $p\times p$ matrix with columns ...
user avatar
1 vote
1 answer
41 views

Projecting onto the space of Upper-Triangular-ish Matrices

I want to solve the independent component analysis (ICA) problem for a single time domain channel of input. The problem is, in my formulation, the mixing matrix has a special structure. I want to know ...
user avatar
  • 1,011
1 vote
0 answers
71 views

How to prove that eig($A_1$)+eig($A_2$)=eig($A'$)?

Given $3$ matrices: $$\begin{aligned} A_1 &= \frac12 \left( X'X'^T + CX'X'^T + X'X'^TC + CX'X'^TC \right)\\ A_2 &= \frac12 \left( X'X'^T - CX'X'^T - X'X'^TC + CX'X'^TC \right)\\ A' &= X'X'^...
user avatar
1 vote
1 answer
39 views

Principal component analysis for a set of p-adic numbers?

I have been exploring machine learning algorithms that use p-adic metrics instead of Euclidean ones. Generally, where a Euclidean algorithm operates over $\mathbb{R}^n$, I have found that $\mathbb{Q}...
user avatar
1 vote
1 answer
81 views

Rewriting the objective function of PCA

Let $X \in \Bbb R^{m \times n}$ and $V \in \Bbb R^{m\times d}$. Prove that $$ \mathcal{F}_{\mathrm{PCA}}(Y)=\sum_{i=1}^{n}\left\|y_{i}-\frac{1}{n} \sum_{j=1}^{n} y_{j}\right\|_{2}^{2} = \operatorname{...
user avatar
0 votes
1 answer
41 views

PCA with mean centered data

see attachment Consider an $n \times p$ data matrix $\mathbf X$ with mean-centered data (i.e. each variable has mean zero). And suppose that we perform PCA on $\mathbf X$, obtaining principal ...
user avatar
  • 11
0 votes
1 answer
98 views

Is PCA at odds with stability?

For simplicity, let's consider a 2D case only. I am measuring coordinates of say some 1000 data points lying on a regular 2D cartesian plane. The recorded $(x,y)$ values are placed as columns of ...
user avatar
0 votes
2 answers
131 views

Am I Misunderstanding Principal Component Analysis (PCA)?

Principal Component Analysis (PCA) has routinely caused me to question my understanding of mathematics, particularly linear algebra. Once again, PCA is present and I would like to engage to the ...
user avatar
0 votes
0 answers
15 views

How to use two SPSS factor scores to generate one social vulnerability index value

I am trying to produce a single value representing social vulnerability based on 6 z-score normalized demographic variables for 55 geographic areas. I have successfully done a factor analysis on the 6 ...
user avatar
1 vote
0 answers
34 views

Intuition behind performing principal components analysis on the linear approximation of a time-series from another time-series.

I was reading a recent paper and was trying to understand the novel factor analysis method that they introduce. I am not terrific at linear algebra so I was hoping to get some intuition behind what ...
user avatar
1 vote
1 answer
62 views

Approximating large quadratic optimization problems

For some positive-definite matrix $A \in \mathbb{R}^{K \times K}$ I want to solve the quadratic optimization problem $$\max_{x\in [0,1]^K} x^T A x \\ \text{s.t.} \\ \sum_{i=1}^{K}x_{i}=1$$ The problem ...
user avatar
0 votes
0 answers
36 views

Maximizing the variance of projected data

Show that this expression can be written as: I have tried multiple approaches like going through a backward proof by taking individual norms and trying to reconstruct but I am unable to find a clue. ...
user avatar
0 votes
1 answer
77 views

Maximizing the Lagrangian with respect to vector - help solve

I need help solving this following equation, a Lagrangian problem that I encountered during my studies in principal component analysis (PCA). One should maximize the variance with respect to the first ...
user avatar