0
$\begingroup$

Let's say I have the following matrix $A$:

$$A = \begin{bmatrix}1 & 1 & 1\\ 1 & 1 & 1\\ 1 & 1 & 1 \end{bmatrix}$$

and I row reduce it to $A'$:

$$A' = \begin{bmatrix}1 & 1 & 1\\ 0 & 0 & 0\\ 0 & 0 & 0 \end{bmatrix}$$

Correct me if I'm wrong, but elementary row operations show us if the rows are linearly dependant/independant, not if the columns are linearly dependant/independant. They don't affect the row space $R(A)$. but they do change the column space $C(A)$.

$C(A) = \left\{\lambda \begin{bmatrix}1\\1\\1\end{bmatrix} \ni \lambda \in \mathbb{R}\right\}$ whereas $C(A') = \left\{\lambda \begin{bmatrix}1\\0\\0\end{bmatrix} \ni \lambda \in \mathbb{R}\right\}$

Now since elementary row operations change the columns space $C(A)$, how can we find the linearly independent columns of $A$ and express $C(A)$ as linear combinations of those linearly independent columns?

Would we have to revert to elementary column operations (if there is such a thing I've never seen it before), to column reduce $A$ and find it's linearly independent columns?

Furthermore, what is the best way/method to find the columns space $C(A)$? Is it better to find the linearly independent columns and express $C(A)$ in terms of those, or do you just take the linear combinations of all the columns, and simplify from there?

$\endgroup$
1
$\begingroup$

You can obtain a basis for the column space by performing row operations.

After you obtain the row-echelon form, note down those columns that are pivotal columns.

Look at the original matrix, those columns should be linearly independent.

$\endgroup$
1
$\begingroup$

Fundamental Theorem of Linear Algebra

A matrix $\mathbf{A} \in \mathbb{C}^{m\times n}_{\rho}$ induces for fundamental subspaces: $$ \begin{align} % \mathbf{C}^{n} = \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} \oplus \color{red}{\mathcal{N} \left( \mathbf{A} \right)} \\ % \mathbf{C}^{m} = \color{blue}{\mathcal{R} \left( \mathbf{A} \right)} \oplus \color{red} {\mathcal{N} \left( \mathbf{A}^{*} \right)} % \end{align} $$


Reduced row echelon forms

Perhaps we can sort things out by looking at a close variation of this problem on a matrix which is not square: $$ \mathbf{A} = \left[ \begin{array}{ccc} 1 & 1 & 1 \\ 1 & 1 & 1 \\ \end{array} \right] $$

Row space

$$ \begin{align} \left[ \begin{array}{c|c} \mathbf{A} & \mathbf{I}_{2} \\ \end{array} \right] & \mapsto \left[ \begin{array}{c|c} \mathbf{E_{A}} & \mathbf{R} \\ \end{array} \right] \\ % \left[ \begin{array}{cc|cc} 1 & 1 & 1 & 0 \\ 1 & 1 & 0 & 1 \\ \end{array} \right] % & \mapsto % \left[ \begin{array}{cc|rr} \boxed{1} & 1 & 1 & 0 \\\hline 0 & 0 & \color{red}{-1} & \color{red}{1} \\ \end{array} \right] % \end{align} $$ The boxed pivot marks the sole fundamental column.

The subspace decomposition for the row space is $$ \mathbb{C}^{2} = \mathcal{R}\left( \mathbf{A} \right) \oplus \mathcal{N}\left( \mathbf{A}^{*} \right) = \text{span } \left\{ \, \color{blue}{\left[ \begin{array}{c} 1 \\ 1 \end{array} \right]} \, \right\} \oplus \text{span } \left\{ \, \color{red}{\left[ \begin{array}{r} -1 \\ 1 \end{array} \right]} \, \right\} $$

Column space

$$ \begin{align} \left[ \begin{array}{c|c} \mathbf{A^{*}} & \mathbf{I}_{3} \\ \end{array} \right] & \mapsto \left[ \begin{array}{c|c} \mathbf{E_{A}^{*}} & \mathbf{R} \\ \end{array} \right] \\ % \left[ \begin{array}{cc|ccc} 1 & 1 & 1 & 0 & 0 \\ 1 & 1 & 0 & 1 & 0 \\ 1 & 1 & 0 & 0 & 1 \\ \end{array} \right] % & \mapsto % \left[ \begin{array}{cc|rrc} \boxed{1} & 1 & 1 & 0 & 0 \\\hline 0 & 0 & \color{red}{-1} & \color{red}{1} & \color{red}{0} \\ 0 & 0 & \color{red}{-1} & \color{red}{0} & \color{red}{1} \\ \end{array} \right] % \end{align} $$ The boxed pivot marks the sole fundamental column.

The subspace decomposition for the column space is $$ \mathbb{C}^{3} = \mathcal{R}\left( \mathbf{A} \right) \oplus \mathcal{N}\left( \mathbf{A}^{*} \right) = \text{span } \left\{ \, \color{blue}{\left[ \begin{array}{c} 1 \\ 1 \\ 1 \end{array} \right]} \, \right\} \oplus \text{span } \left\{ \, \color{red}{\left[ \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \right]}, % \, \color{red}{\left[ \begin{array}{r} -1 \\ 0 \\ 1 \end{array} \right]} \, \right\} $$


One way to find spans for the subspaces is to use...

Singular value decomposition

The power of the SVD is that it provides orthonormal spans for all subspaces.

Every matrix $$ \mathbf{A} \in \mathbb{C}^{m\times n}_{\rho} $$ has a singular value decomposition

$$ \begin{align} \mathbf{A} &= \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \\ % &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}_{\rho\times \rho} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] \\ % &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cccc|cc} \sigma_{1} & 0 & \dots & & & \dots & 0 \\ 0 & \sigma_{2} \\ \vdots && \ddots \\ & & & \sigma_{\rho} \\\hline & & & & 0 & \\ \vdots &&&&&\ddots \\ 0 & & & & & & 0 \\ \end{array} \right] % V \left[ \begin{array}{c} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ \color{red}{\mathbf{V}_{\mathcal{N}}}^{*} \end{array} \right] \\ % & = % U \left[ \begin{array}{cccccccc} \color{blue}{u_{1}} & \dots & \color{blue}{u_{\rho}} & \color{red}{u_{\rho+1}} & \dots & \color{red}{u_{n}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V \left[ \begin{array}{c} \color{blue}{v_{1}^{*}} \\ \vdots \\ \color{blue}{v_{\rho}^{*}} \\ \color{red}{v_{\rho+1}^{*}} \\ \vdots \\ \color{red}{v_{n}^{*}} \end{array} \right] % \end{align} $$

The column vectors are orthonormal basis vectors: $$ \begin{align} % R A \color{blue}{\mathcal{R} \left( \mathbf{A} \right)} &= \text{span} \left\{ \color{blue}{u_{1}}, \dots , \color{blue}{u_{\rho}} \right\} \\ % R A* \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} &= \text{span} \left\{ \color{blue}{v_{1}}, \dots , \color{blue}{v_{\rho}} \right\} \\ % N A* \color{red}{\mathcal{N} \left( \mathbf{A}^{*} \right)} &= \text{span} \left\{ \color{red}{u_{\rho+1}}, \dots , \color{red}{u_{m}} \right\} \\ % N A \color{red}{\mathcal{N} \left( \mathbf{A} \right)} &= \text{span} \left\{ \color{red}{v_{\rho+1}}, \dots , \color{red}{v_{n}} \right\} \\ % \end{align} $$ The decomposition distinctly reflects the reduction operations: $$ % \begin{align} % \mathbf{A} = \left[ \begin{array}{ccc} 1 & 1 & 1 \\ 1 & 1 & 1 \\ \end{array} \right] &= % U \left[ \begin{array}{cc} \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}} \end{array} \right] % Sigma \left[ \begin{array}{cc} \mathbf{S}_{1\times 1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] % V* \left[ \begin{array}{cc} \color{blue}{\mathbf{V}_{\mathcal{R}}} & \color{red}{\mathbf{V}_{\mathcal{N}}} \end{array} \right]^{*} \\ % &= % \frac{1}{\sqrt{2}} \left[ \begin{array}{cr} \color{blue}{1} & \color{red}{-1} \\ \color{blue}{1} & \color{red}{1} \\ \end{array} \right] % \left[ \begin{array}{c|cc} \sqrt{6} & 0 & 0 \\\hline 0 & 0 & 0 \\ \end{array} \right] % \left[ \begin{array}{ccc} % c1 \frac{1}{\sqrt{3}} \color{blue}{\left[ \begin{array}{r} 1 \\ 1 \\ 1 \end{array} \right]} % c2 \frac{1}{\sqrt{2}} \color{red}{\left[ \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \right]} % c3 \frac{1}{\sqrt{2}} \color{red}{\left[ \begin{array}{r} -1 \\ 0 \\ 1 \end{array} \right]} % \end{array} \right]^{*} % \end{align} % $$

$\endgroup$
0
$\begingroup$

The columns of $A$ are the rows of $A^T$. If you are uncomfortable with column operations, you can row-reduce $A^T$ to a matrix $B$ and then transpose back.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.