Questions tagged [principal-component-analysis]

Principal component analysis (PCA) is a linear dimensionality reduction technique. It reduces a multivariate dataset to a smaller set of constructed variables preserving as much information (as much variance) as possible. These variables, called principal components, are linear combinations of the input variables.

Filter by
Sorted by
Tagged with
0
votes
0answers
26 views

Why can $X$ be approximated as $X V V^T$?

I was given the statement If $\vec{x_1}, \dots, \vec{x_n}$ are close to a k-dimensional subspace $\mathbb{V}$ with orthonormal basis $V \in \mathbb{R}^{d \times k}$, then the data matrix can be ...
0
votes
2answers
45 views

Am I Misunderstanding Principal Component Analysis (PCA)?

Principal Component Analysis (PCA) has routinely caused me to question my understanding of mathematics, particularly linear algebra. Once again, PCA is present and I would like to engage to the ...
0
votes
0answers
29 views

Questions on robust PCA

I am reading a paper on robust PCA by Candès et al. In section 2.3, the authors said $\| \mathcal P_{\Omega} \mathcal P_T \| < 1$ is equivalent to $\Omega \cap T = \{0\}$. Can someone please ...
0
votes
0answers
8 views

How to use two SPSS factor scores to generate one social vulnerability index value

I am trying to produce a single value representing social vulnerability based on 6 z-score normalized demographic variables for 55 geographic areas. I have successfully done a factor analysis on the 6 ...
0
votes
0answers
3 views

Angle Preserve Dimension Reduction

PCA is known to be one of the most-used Dimension Reduction algorithms. But does it preserve angles between vectors? If no, are there any algorithms that do preserve angles and (what is the same) ...
1
vote
0answers
28 views

Intuition behind performing principal components analysis on the linear approximation of a time-series from another time-series.

I was reading a recent paper and was trying to understand the novel factor analysis method that they introduce. I am not terrific at linear algebra so I was hoping to get some intuition behind what ...
1
vote
1answer
43 views

Approximating large quadratic optimization problems

For some positive-definite matrix $A \in \mathbb{R}^{K \times K}$ I want to solve the quadratic optimization problem $$\max_{x\in [0,1]^K} x^T A x \\ \text{s.t.} \\ \sum_{i=1}^{K}x_{i}=1$$ The problem ...
0
votes
0answers
34 views

Finding Orthonormal Basis using SVD and comparing it with Gram-Schmidt shows different result

I was trying to find the orthonormal basis for the column space of the following matrix "A" \begin{pmatrix} -1 & -1 & 2 & 3 \\ -1 & 1 & -3 & -4 \\ 2 & -2 & 5 ...
0
votes
0answers
7 views

Principal component Eigen values

I am learning principal component : In Below question I am facing problem to calculate the eigen values and eigen vector. Is there any short method to determine that. Consider the following covariance ...
0
votes
0answers
34 views

Maximizing the variance of projected data

Show that this expression can be written as: I have tried multiple approaches like going through a backward proof by taking individual norms and trying to reconstruct but I am unable to find a clue. ...
-1
votes
1answer
58 views

Maximizing the Lagrangian with respect to vector - help solve

I need help solving this following equation, a Lagrangian problem that I encountered during my studies in principal component analysis (PCA). One should maximize the variance with respect to the first ...
0
votes
0answers
15 views

Probability of PCA parameters?

Let's say I have a PCA model computed using SVG and have kept $L$ parameters, so I have matrix of mean values $\mu$, a matrix with the components $W$ and a vector of standard deviations $s$(?). I can ...
0
votes
0answers
11 views

Low Loadings in PCA

I am new user of this forum, and I am not sure whether I posted my question appropriate place, sorry for that. I am running PCA for determining gentrification score for census tracts. Actually, I am ...
1
vote
1answer
35 views

Projection of $\mathbf{X}$ in the direction of $\mathbf{a}$: $\mathbf{a}^T \mathbf{X} = \sum_{j = 1}^d a_j X_j$

I am currently studying principal component analysis in statistics. PCA uses the "projection of $\mathbf{X}$ in the direction of $\mathbf{a}$": $$\mathbf{a}^T \mathbf{X} = \sum_{j = 1}^d ...
0
votes
0answers
21 views

Understanding orthogonal vector projections

I am trying to understand the linear algebra of PCA, and specifically what does it mean to be the $i$th principal component of some random vector $x$. I do know that this refers to the coordinate of $...
0
votes
0answers
15 views

EigenValue decomposition component matrices meanings

I am on a Principal component analysis which is helping me with calculation problem : i want to use the eigenValue decomposition on a certain covariance matrix to ease a calculation, but stuck with ...
1
vote
0answers
13 views

How to put these metrics into a formula to describe their dependency on each other?

I'm not a mathematician - far from that, but in a current task I think about, I come to idea, that the solution can be found only on mathematics way. I have five metrics, and I want to get a formula, ...
0
votes
0answers
50 views

Simple, intuitive high school level explanation of spectral theorem and the covariance matrix

Ive been trying to understand spectral theorem ( a symmetric matrix has n independent, orthogonal eigenvectors ), but the proof seems to be way above my level. Similarly, ive been trying to realize ...
1
vote
0answers
19 views

expectation of covariance matrix/singular decomposition matrix

So $X_1,\dots,X_n$ are i.i.d. d-variate random vectors (with finite second moments) and $X=(X_1,\dots,X_n)^T$ not necessarily centred. Now I'm trying to find $E(S_n(X))$, where $S_n(X)$ is the sample ...
1
vote
1answer
29 views

Question about the FG algorithm (convex optimization) (Flury 1984)

I am currently trying to understand the FG algorithm which was presented by Flury in 1984. You can find here a description of the algorithm : https://www.researchgate.net/publication/...
0
votes
0answers
12 views

PCA - linear combination of dimensions reduced vectors combination doesn't equal the reduced result of the linear combination of the high dim vectors

Suppose I perform PCA on a set of vectors x1,...,xn How come PCA(a1 * x1+a2 * x2) != a1 * PCA(x1) + a2 * PCA(x2)? Example Code in Python - ...
2
votes
1answer
95 views

Harris corner detection vs. PCA

A lot of classes shows that Harris corner detector is actually PCA (example, page 5), but I'm having trouble with that since Harris doesn't have mean subtraction in the algorithm (not in the paper and ...
0
votes
1answer
40 views

Start with PCA and multiple regression or Start with multiple regression and PCA

I would like to know something easy but very important. Imagine I have a database with 0 NA, a perfect database who has been clean. And I have to do a PCA on this database. This datebase got a lot ...
3
votes
1answer
43 views

Why does singular value decomposition simultaneously diagonalize a symmetric matrix and its square?

So I took an online course on machine learning and in this course the instructor said that the eigenvectors of a covariance matrix (for principal components analysis) can be computed by a singular ...
0
votes
0answers
23 views

Meaning of eigenvalues and eigenvectors in the context of PCA

I'm trying to work through the relationship between eigenvectors/values and PCA, but I'm getting confused. I've been referencing Steve Brunton's very helpful video, here. When we take the ...
0
votes
1answer
30 views

Using symmetry of the dot product, how can I prove this?

Context of this question is projection perspective of principal component analysis, suppose we have orthonormal basis vectors $b_1, ... , b_m$ of the principal subspace where we project a data point $...
1
vote
2answers
78 views

Maximising the variance in PCA - Trace Proof

Quite urgent request for a proof here as my professor has not run through it. Scenario is the PCA problem where we are trying to maximise the variance in the latent space. I want to prove the ...
0
votes
0answers
23 views

Applying PCA to Coefficients of Polynomial Regression

hi guys, I'm looking for a hint or search term to find some details for the following idea: Record several (e.g. 10) curves with plenty of data points Perform polynomial curve fitting: $y=\beta_0 + \...
2
votes
1answer
38 views

Why is the first eigenvalue on the principal component always superior to the second eigenvalue?

I'm learning PCA and I wanted to know why the first eigenvalue on the principal component is always superior at second eigenvalue. Thanks for helping me Ps: I tried to unterstand before asking this ...
0
votes
0answers
25 views

What can I learn about the columns with highest variance of a matrix $M\approx L^TR$ from looking at $L$?

I have a high-dimensional, symmetric matrix $M\in\mathbb{R}^{d\times d}$ , which is factorized by two matrices $L, R\in \mathbb{R}^{n\times d}$ : $L^TR\approx M$, where $n$ is much smaller than $d$. ...
0
votes
0answers
43 views

Proof of the Frobenius Norm is the trace

My professor presented the following in class and I have not been able to construct the proof and wondered if someone could explain this please? This is looking at the reconstruction error for PCA. $W$...
1
vote
0answers
13 views

Expressing the discriminant function in terms of dimensionality and complexity.

If we're working with several number of features, then we deal with a high dimensional space. I found out that in particular he number of my cofficients is $O(D^M)$ where $D$ is the number of features ...
0
votes
0answers
30 views

Principal Component Analysis of a Linear combination

I am trying prove the following statement. Let $Z_i=AX_i+b$ for $i \in \{1,2,3,4,..,n\}$ where $Z_i\in\mathcal{R}^{p}$ then then Principal Components for $Z_i$ are the same as those for $X_i$ after ...
0
votes
1answer
18 views

Explanation regarding eigenvectors and eigenvalues

In PCA algorithm, $C$ the covariance matrix is defined as $\frac1{N-1}XX^T$, where $X$ is a matrix of size $d\times n$. Also, the rank of $X$ is $\min(d-1,n)$. (Lets assume) Now, since $C$ is ...
0
votes
0answers
21 views

Principal Component Analysis (interpreting results)

I've just started learning PCA. I am interested in principal components with eigenvalues greater than 1 (using Kaiser criteria). So the y-axis on the graph should be an eigenvalue (1, 2, 3, etc.) But ...
0
votes
0answers
25 views

Is only one SVD computation enough to perform PCA on a matrix and its transpose?

SVD is used in PCA in order to get the mapping to lower dimensions. Is it enough to perform only one SVD in order to get the PCA for the original matrix AND its transpose, considering that the SVD of ...
1
vote
0answers
32 views

How to choose W in Pearson's PCA derivation?

Given that $U\Lambda V^t$ is the SVD decomposition of Y $(Y=U\Lambda V^t)$, and we have to choose k singular values ($\sigma _i$) for the sake of dimensionality reduction, prove that: $trace(Y^tWW^tY)=...
1
vote
0answers
28 views

Derivation of optimization objective in PCA (Principal Components Analysis)

Suppose that we have $\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n$ centered points in $m$ dimensional space. Let $\mathbf{v}$ denote the unit vector along which we project our $\mathbf{x}$'s. The ...
2
votes
0answers
47 views

What is the term for two matrices sharing the same sets of singular vectors?

Consider the singular value decompositions $A=U_A\Sigma_AV_A^T$ and $B=U_B\Sigma_BV_B^T$. Is there a word that describes the relation between $A$ and $B$ when they have the same left and/or right ...
3
votes
1answer
20 views

Principal Component Analysis of variables $x$, $x^2$, $x^3$?

I know the working of PCA, but was baffled by question asked in the interview that: Principal Component Analysis of variables $x$, $x^2$, $x^3$. How can we derive it mathematically? Do we need to ...
-1
votes
1answer
32 views

Using principal component analysis to reduce dimension, how can we know the distance between data points is reduced and features are uncorrelated?

Using PCA, if we reduce the dimension of a dataset $x_1, \dots, x_n \in \mathbb{R}^d$ of mean zero, then we can get a dimensionally reduced dataset $y_1, \dots , y_n \in \mathbb{R}^k$, for some $1\leq ...
0
votes
0answers
16 views

What is the difference between principal component analysis and correspondence analysis when applied to row/column profiles of contingency tables?

I am dealing with a dataset of nominal data. Each category is essentially a feature of a binary classifier. My goal is to find the features or potential combinations of them that are most correlated ...
0
votes
1answer
17 views

Principal component analysis Covariance derivation

How is it determined that the Covariance matrix has Eigenvectors which are in the direction of largest variation of a data set? I suppose to derive this you would maybe use regression line for the ...
0
votes
0answers
12 views

Principal component analysis loadings

I am confused about the loadings in principal component analysis, are the loadings simply the ratio of the variables representative of a principal component or are they coefficient values for the ...
0
votes
0answers
8 views

Getting rid of Multicollinearity in Regression Analysis through Factor Analysis

Multicollinearity exists when two or more independent variables used in regression are correlated. Applying a regression analysis is not advised if multicollinearity exists as it increases the ...
0
votes
0answers
33 views

Understanding PCA Transformation

I have a question regarding the last step of PCA; transforming the data matrix X into the new subspace. After doing SVD with X = UEV* and computing ...
0
votes
0answers
15 views

What are the principal components representative of?

What are the principal components representative of? I know they are on the direction of most variation but what are their units? If you capture the direction of most variation of the initial data, ...
0
votes
2answers
31 views

Find the principal components of a $4\times 4$ matrix

I want to find the principal components of the matrix $$ A=\pmatrix{1,\rho,\rho, \rho\\\rho,1,\rho,\rho\\\rho,\rho,1,\rho\\ \rho,\rho,\rho,1} $$ I assume $\rho$ is correlation. To find the principal ...
0
votes
0answers
20 views

Intuition behind eigen vectors

Why do we use the eigen vectors corresponding to the low eigen values of the laplacian matrix in most of the applications? Do the eigen vectors of the Laplacian matrix represent only frequencies or ...
2
votes
0answers
27 views

PCA dimensions question

I read this quote in some lectures about PCA: " Also, since n points in a p-dimensional space defines a linear subspace whose dimension is at most n−1, we would find that p−n+ 1 eigenvalues are ...