Linked Questions

48
votes
3answers
101k views

How do I exactly project a vector onto a subspace?

I am trying to understand how - exactly - I go about projecting a vector onto a subspace. Now, I know enough about linear algebra to know about projections, dot products, spans, etc etc, so I am not ...
4
votes
3answers
2k views

Is the least-squares solution unique?

I am looking for a line closest to $(-5, -2)$, $(-2, 0)$, $(-1, 0)$, $(2, 3)$, $(5, 4)$ using the least square solution. So I set the line as $$ax+by+c=0$$ let $a=1$ (where $a$ is not $0$ obviously) ...
2
votes
2answers
3k views

Pseudo inverse of a singular value decomposition SVD is equal to its “real” inverse for a square matrix?

I was reading this book on numeric linear algebra and it said pseudo inverse of a singular value decomposition (SVD) is equal to it's "real" inverse for a square matrix. It said it is quite clear that ...
7
votes
2answers
1k views

Roles of $\bf A^TA$ ($\text {A transpose A}$) matrices in orthogonal projection

$\bf A^TA$ forms (or equivalently (?) positive semidefinite matrices, or more particularly, covariance matrices($\bf \Sigma$)) are linked in practice to many operations in which data points are ...
2
votes
3answers
947 views

Overdetermined System Ax=b

Let's say we have the following system of equations: \begin{equation} A{\bf x}={\bf b} \qquad (1) \end{equation} where $A$ is $N \times 4$, $\mathbf{x}$ is $4 \times 1$ (unknowns) and ${\bf b}$ is $...
1
vote
2answers
2k views

Checking my understanding of projection onto subspace vs least square approximation

I have trouble understanding the topic of projection vs. least square approximation in an Introductory Linear Algebra class. I know this question has already been asked (Difference between orthogonal ...
1
vote
1answer
1k views

Least squares solutions and the orthogonal projector onto the column space

Question: Suppose that 4 vectors, v1, v2, v3, and v4 are given in $R^{6}$ and we don’t know whether or not these vectors are linearly independent. Explain how you would find the (projection) matrix ...
1
vote
3answers
936 views

Why $\Sigma$ in the SVD of a matrix $A$ is invertible? [closed]

I know singular value decomposition of a matrix $A$ is $A=U\Sigma(V^T)$. And if we find to $x$ in $Ax=b$ we can use $A=U\Sigma V^T$ to get $x=V\Sigma'(U^T)b$. But could somebody explain why $\Sigma$ ...
1
vote
1answer
667 views

Why is the projection matrix $P = A(A^T A)^{-1} A^T$ left-multiplied by $A$?

Consider a vector space $V$ and its (orthogonal) subspaces $W$ and $U$. If $A$ is a matrix representing the linear map $T: V \rightarrow W$, and we want to project an element of $U$ onto $W$, why is ...
3
votes
1answer
822 views

Least square matrix form will fail, if the inverse property not satisfied?

In the matrix form of least squares , the inverse of ( X transpose X ) we are calculating . So, what if that matrix does not posses inverse properties. I mean what if it is not invertible ? Sorry if ...