# Change of basis of the kernel of a rectangular matrix

Suppose that I have a linear transformation $$T: \mathbb{R}^n \rightarrow \mathbb{R}^m$$ such that $$T$$ can be written as an $$m \times n$$ matrix. Let $$k = \dim\ker T$$, where $$k > 0$$. Thus, $$\ker T \cong \mathbb{R}^k$$, and any basis of $$\ker T$$ is a set consisting of $$k$$ linearly-independent vectors of $$\ker T$$. Nevertheless, the vectors of $$\ker T$$ are written (in the standard basis of $$\mathbb{R}^n$$) as a column of $$n$$ scalars (elements of $$\mathbb{R}$$).

Given a vector $$v \in \ker T$$ written in terms of a basis $$\mathcal{B}$$ of $$\ker T$$, I want to write $$v$$ in terms of another basis $$\mathcal{B}'$$.

I know that this typically entails computing the change-of-basis matrix (and its inverse), but I am mostly finding discussions and examples of this involving scenarios where the number of basis vectors equals the number of scalars in the column that "constitutes" the vector $$v$$ (in the standard basis).

Am I seemingly running into an issue here because I am trying to think of $$v$$ in terms of the basis of $$\mathbb{R}^n$$ (that is, $$[v]_{\mathcal{E_n}}$$, where $$\mathcal{E}_n$$ is the standard basis of $$\mathbb{R}^n$$)? If I instead consider $$[v]_{\mathcal{B}}$$ and $$[v]_{\mathcal{B}'}$$, then $$v$$ can be expressed as columns of $$k$$ scalars, so that the typical square change-of-basis matrix formalism can be used, correct? Though it seems I would need to know how to express the basis vectors of $$\mathcal{B}$$ in terms of those of $$\mathcal{B}'$$ (or vice versa) in order to generate this change-of-basis matrix in the first place. Instead, I know how to express the elements of $$\mathcal{B}$$ and $$\mathcal{B}'$$ in terms of $$\mathcal{E}_n$$.

Perhaps I can generate two rectangular change-of-basis matrices, $$P_{\mathcal{E}_n \leftarrow \mathcal{B}}$$ and $$P_{\mathcal{B'} \leftarrow \mathcal{E}_n}$$ such that their product $$P_{\mathcal{B'} \leftarrow \mathcal{B}} = P_{\mathcal{B'} \leftarrow \mathcal{E}_n} P_{\mathcal{E}_n \leftarrow \mathcal{B}}$$ is a square matrix that can be inverted? I believe that I can obtain $$P_{\mathcal{E}_n \leftarrow \mathcal{B}}$$ and $$P_{\mathcal{E}_n \leftarrow \mathcal{B}'}$$ by writing down the vectors of $$\mathcal{B}, \mathcal{B}'$$ in the standard basis of $$\mathbb{R}^n$$, but how can I obtain $$P_{\mathcal{B'} \leftarrow \mathcal{E}_n}$$ without having to resort to computing an "inverse" for a rectangular matrix?

If someone could explicitly walk through the steps needed to do this change of basis, that would be greatly appreciated. Thank you!

• Your thought is correct. If you change from the standard basis to a given basis $B$, you obtain the base change matrix by writing the basis vectors of $B$ in a matrix. From this you can compute everything, just write down the base change diagram. – James Feb 27 at 6:28
• @James Thanks for the comment. I have added another thought in the penultimate paragraph of my post. I believe I can obtain the change-of-basis matrices from $\mathcal{B},\mathcal{B}'$ to $\mathcal{E}_n$, but I am not sure how to "invert" one of those matrices to go from $\mathcal{E}_n$ to $\mathcal{B}'$, as seems to be necessary to go from $\mathcal{B}$ to $\mathcal{B}'$. Thanks! – AnInquiringMind Feb 27 at 13:23

You’re on the right track. If you were working with $$\mathbb R^n$$, you might assemble the elements of $$\mathcal B$$ into the columns of the matrix $$B = P_{\mathcal E_n\leftarrow\mathcal B}$$ and the elements of $$\mathcal B'$$ into $$B' = P_{\mathcal E_n\leftarrow\mathcal B'}$$. The change-of-basis matrix $$P_{\mathcal B'\leftarrow\mathcal B}$$ is then the solution to the equation $$B'X=B$$, and since $$B'$$ is full rank, that’s just $$X={B'}^{-1}B$$. Conceptually, $$P_{\mathcal B'\leftarrow\mathcal B}$$ first maps from $$\mathcal B$$ to the standard basis, and then from that to $$\mathcal B'$$.
What if we try the same thing with the two bases of $$\ker T$$, expressed as elements of $$\mathbb R^n$$? Well, the equation $$B'X=B$$ is still valid, but since $${B'}^{-1}$$ doesn’t exist we can’t simply multiply both sides by it as we did above. However, $$B'$$ does have full column rank, so it has the left inverse $$({B'}^TB')^{-1}{B'}^T$$. (That this is reminiscent of the formula for the least-squares solution to a system of linear equations is no coincidence.) Therefore, our change-of-basis matrix is $$P_{\mathcal B'\leftarrow\mathcal B}=({B'}^TB')^{-1}{B'}^TB.$$ This is consistent with the calculation in the first paragraph: if $$B'$$ is invertible, this expression reduces to $${B'}^{-1}B$$.
Taking a concrete example, let $$B = \begin{bmatrix}1&1\\1&0\\1&1\\1&0\end{bmatrix}, B' = \begin{bmatrix}2&0\\1&1\\2&0\\1&1\end{bmatrix}.$$ (The elements of $$\mathcal B'$$ are the sum and difference of the elements of $$\mathcal B$$.) Applying the above formula, we get $$P_{\mathcal B'\leftarrow\mathcal B} = \begin{bmatrix}\frac12&\frac12\\\frac12&-\frac12\end{bmatrix}.$$ This looks plausible given how $$\mathcal B'$$ was constructed from $$\mathcal B$$. To check, convert to the standard basis of $$\mathbb R^4$$: $$B[a,b]^T = [a+b,a,a+b,a]^T$$ and $$B'P_{\mathcal B'\leftarrow\mathcal B}\begin{bmatrix}a\\b\end{bmatrix} = B'\begin{bmatrix}\frac{a+b}2\\\frac{a-b}2\end{bmatrix} = \begin{bmatrix}a+b\\a\\a+b\\a\end{bmatrix}.$$
• Thank you very much for the nice answer! Just one additional clarification question for you: just as $P_{\mathcal{B} \leftarrow \mathcal{E}_n} = \left( P_{\mathcal{E}_n \leftarrow \mathcal{B}} \right)^{-1}$ in the square matrix case, does $P_{\mathcal{B} \leftarrow \mathcal{E}_n} = \left( P_{\mathcal{E}_n \leftarrow \mathcal{B}} \right)_L^{-1}$, where the subscript $L$ denotes left inverse as defined above, in this case of a rectangular matrix with full column rank? – AnInquiringMind Feb 28 at 2:18
• @AnInquiringMind It’s a projection onto the column space of $B'$. This is the identity map on that subspace, so I guess you could consider it a change-of-basis matrix if you restrict the domain accordingly. – amd Feb 28 at 3:13
• You wrote $B'$ in your comment. Just want to make sure: in my comment, I only referenced $\mathcal{B}$ and $\mathcal{E}_n$, so I think you meant $B$, right? – AnInquiringMind Mar 1 at 1:57