1
$\begingroup$

Question: Suppose that 4 vectors, v1, v2, v3, and v4 are given in $R^{6}$ and we don’t know whether or not these vectors are linearly independent. Explain how you would find the (projection) matrix which projects onto the subspace S = span {v1, v2, v3, v4}.

Is it correct to say:

First find the basis of S to obtain a linearly independent columns of a matrix A that will span S. Then use the basis vectors as the columns of A. To find the projection matrix we use $P=A(A^{T}A)^{−1}A^{T}$

So I had asked this question somewhere and someone suggested in using least squares. I am assuming because A is 4x6. Then the projection matrix would just be P=Ax. But, if I try finding the least square, x, I would need a $b$ for the equation $A^{T}Ax=A^{T}b$.

$\endgroup$
1
$\begingroup$

A sample problem like this is explored in Solve an overdetermined system of linear equations.

Start with the linear system $\mathbf{A}x = b$ where $$ \mathbf{A}\in\mathbb{C}^{m \times n}_{\rho}, \quad x \in\mathbb{C}^{n}, \quad b \in\mathbb{C}^{m}. $$ The existence of the singular value decomposition is guaranteed: $$ \mathbf{A} = \mathbf{U}\, \Sigma \, \mathbf{V}^{*} = \underbrace{\left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right]}_{m\times m} % \underbrace{\left[ \begin{array}{cc} \mathbf{S}_{\rho \times \rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right]}_{m\times n} % \underbrace{\left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right]^{*}}_{n\times n} . $$ $\color{blue}{Range\ space}$ entities are colored in blue, $\color{red}{nullspace}$ in red. From that we construct the Moore-Penrose pseudoinverse $$ \mathbf{A}^{\dagger} = \mathbf{V}\, \Sigma^{\dagger} \mathbf{U}^{*}. $$ The four fundamental unitary projectors are $$ \begin{align} % \mathbf{P}_\color{blue}{\mathcal{R}\left( \mathbf{A} \right)} &= \mathbf{A}\mathbf{A}^{\dagger} & % \mathbf{P}_\color{red}{\mathcal{N}\left( \mathbf{A}^{*} \right)} &= \mathbf{I}_{m} - \mathbf{A}\mathbf{A}^{\dagger} \\ % \mathbf{P}_\color{blue}{\mathcal{R}\left( \mathbf{A}^{*} \right)} &= \mathbf{A}^{\dagger}\mathbf{A} & % \mathbf{P}_\color{red}{\mathcal{N}\left( \mathbf{A} \right)} &= \mathbf{I}_{n} - \mathbf{A}^{\dagger}\mathbf{A} \\ % \end{align} $$ You are asking about $$ \mathbf{P}_\color{blue}{\mathcal{R}\left( \mathbf{A} \right)} = \mathbf{A}\mathbf{A}^{\dagger} = \mathbf{A}\left( \mathbf{A}^{*}\mathbf{A}\right)^{-1}\mathbf{A}^{*}. $$

The connection between the pseudoinverse and the normal equations solutions is shown in SVD and linear least squares problem

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.