Finding inverse of a matrix geometrically

Question

I completed watching 3b1b's linear algebra series, where matrices were defined as describing linear transformation of basis vectors. The inverse of a matrix reverses the effect of the transformation applied, thus making no changes.

I want to know how I can visually/geometrically interpret/find the inverse of matrix, given the first transformation is applied.(I'm able to visualize them if the transformed vectors are also unit vectors but couldn't extend the application to other vectors.)

Answer 1

I think this is an excellent question that merits some thought. In contrast to Victor, I would like to interpret the matrix in a different way which leads to an elegant interpretation of the inverse geometrically.

Given a matrix, $$ \begin{bmatrix} a & b \\ c & d \end{bmatrix}, $$ we can geometrically interpret its action as changing coordinates. Therefore, its inverse should correspond to reverting to our previous coordinates. A necessary geometric idea to visualize this is (possibly non-orthogonal) projection.

An elementary (possibly non-orthogonal) projection can be defined as follows. Suppose we are given two non-zero vectors $r, p \in \mathbf{R}^n$. The vector $p$ is the direction we wish to project onto and $r$ is used to define a linear function $r^T$ from $\mathbf{R}^n$ to $\mathbf{R}$. $r^T$ has null space that defines a hyperplane(a line in $\mathbf{R}^2$, or a plane in $\mathbf{R}^3$, etc.). A useful fact is when a set of points $x$ satisfy $r^Tx = \text{constant}$ the points $x$ look like the null space shifted from the origin. Concretely, this means given any line in $\mathbf{R}^2$ we can shift the line so that it intersects any point on the plane. Now, a (possibly non-orthogonal) projection acts on a point $z$ in the following way: find the hyperplane defined by $r^T$ that $z$ lies in (which is the set of $x$ satisfying $r^Tx = r^Tz$), find the intersection of the points satisfying $r^Tx = r^Tz$ with the line $\text{span}\{p\}$, slide $z$ along this hyperplane to the point where $r^Tx = r^Tz$ intersects $\text{span}\{p\}$. Visual of projections

We deduce a formula from this geometric description by noting that we are looking for an element in $\text{span}\{p\}$ that we denote $\lambda p$. We know $\lambda p$ lies on the same hyperplane as $z$ defined by $r^Tx = r^Tz$ which implies that $r^T(\lambda p) = r^Tz$ which leads us to $\lambda = \frac{r^Tz}{r^Tp}$. Thus the result of our (possibly non-orthogonal) projection is $\frac{pr^T}{r^Tp}z$. We note that $\lambda$ is the important factor that tells us how far along $p$ we are in terms of multiples of $p$. Also, we must add the condition that $r^Tp \neq 0$ so there is no division by zero. This condition geometrically asserts that there is one solution to our problem and that if $r^Tp = 0$ our problem is ill-posed because there are infinitely many points satisfying our definition.

Now back to the original problem of the inverse of a matrix. Let the two columns of the matrix define the coordinate system we want to (possibly non-orthogonally) project onto. As a mental image you should currently have a plane with two axes that need not be perpendicular. We will try to project onto $\begin{bmatrix} a \\ c \end{bmatrix}$ using the aforementioned projections. Given a point we want to slide it along the lines parallel to $\begin{bmatrix} b \\ d \end{bmatrix}$ so that we eventually intersect the span of $\begin{bmatrix} a \\ c \end{bmatrix}$. This means we should choose $r = \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}\begin{bmatrix} b \\ d \end{bmatrix}$, which is simply rotating $\begin{bmatrix} b \\ d \end{bmatrix}$ by ninety degrees counter-clockwise, and $ p = \begin{bmatrix} a \\ c \end{bmatrix}$. In our new coordinate system, the value of our first coordinate (how far along $p$ are we) should be $\lambda$ or how many units of $p$ is the vector. Computing $\lambda$ we find,

$$ \lambda = \frac{r^Tz}{r^Tp} = \frac{1}{ad-bc}\begin{bmatrix} d & -b \end{bmatrix}. $$

This is the first row of the inverse of a $2\times2$ matrix. The second row of the inverse of a $2\times2$ matrix can be computed analogously. For general matrices we can do the exact same thing, however finding $r$ becomes trickier. Can you see why? A clever way to find $r$ uses the determinant, but perhaps that's for another post.

Answer 2

If I understood your problem correctly, I would suggest the following approach:

• First, go to this link and check the transformations that are given there for 2D.

• Next try to write "decompose" your transformation matrix into those types of transformations because it is easier to see the inverse for them.

• Since

Maybe two examples will help. As a first case (the easy one) take the matrix

$$\begin{bmatrix} 0 & 2 \\ 2 & 0\end{bmatrix}$$

It is a basic case, but I emphasizes the approach. I believe that you can see that this matrix can be written as

$$\begin{bmatrix} 2 & 0 \\ 0 & 2\end{bmatrix}\begin{bmatrix} 0 & 1 \\ 1 & 0\end{bmatrix} = \begin{bmatrix} 2 & 0 \\ 0 & 2\end{bmatrix}\begin{bmatrix} \cos(\pi/2) & \sin(\pi/2) \\ \sin(\pi/2) & \cos(\pi/2)\end{bmatrix} $$

which is a rotation by $\pi/2$ and a rescaling by a factor of $2$.

In the second example I will take the matrix

$$\begin{bmatrix} 2 & 1 \\ 0 & 2\end{bmatrix}$$

I see, using the wiki link that this looks like a shear parallel to the $x$ axis so maybe I should take out an identity matrix for the scaling as in the first example such that

$$\begin{bmatrix} 2 & 0 \\ 0 & 2\end{bmatrix}\begin{bmatrix} 1 & 0.5 \\ 0 & 1\end{bmatrix}$$

This indicates that I can use a shear and rescaling for that.

Now you have the transformation decomposed in more basic transforms and you should be able to see step by step what to do in order to go from the initial vector to the transformed one.

Note: It might be useful to take into consideration the existence of the statement $(AB)^{-1} = B^{-1}A^{-1}$.

EDIT:

I will take the general scenario of a matrix

$$A = \begin{bmatrix} a & b \\ c & d\end{bmatrix}$$

from which the following calculations can be made. I mention first that whenever rescaling occurs, I will just use the scaling value for ease, not the identity matrix multiplied by the value. Also I'll put below a list of all the transformations that I will use (and their inverse):

• scaling by $k$: $k \rightarrow k^{-1}$
• stretching by $k$: $\begin{bmatrix} k & 0 \\ 0 & 1\end{bmatrix} \rightarrow \begin{bmatrix} 1/k & 0 \\ 0 & 1\end{bmatrix}$ or $\begin{bmatrix} 1 & 0 \\ 0 & k\end{bmatrix} \rightarrow \begin{bmatrix} 1 & 0 \\ 0 & 1/k\end{bmatrix}$
• shearing by $k$: $\begin{bmatrix} 1 & k \\ 0 & 1\end{bmatrix} \rightarrow \begin{bmatrix} 1 & -k \\ 0 & 1\end{bmatrix}$ or $\begin{bmatrix} 1 & 0 \\ k & 1\end{bmatrix} \rightarrow \begin{bmatrix} 1 & 0 \\ -k & 1\end{bmatrix}$

Going back to the general matrix, first I factored $d$ such that

$$A = \begin{bmatrix} a/d & b/d \\ c/d & 1\end{bmatrix}d$$

Next I try to see what happens if I multiply to the matrix the inverse of a stretching matrix by $a/d$. This gives

$$A = \begin{bmatrix} 1 & b/d \\ c/a & 1\end{bmatrix}\begin{bmatrix} a/d & 0 \\ 0 & 1\end{bmatrix}d$$

Apply the inverse of a shear by $c/a$ on the first matrix from the above formula which implies that the following form is true

$$A = \begin{bmatrix} 1-(bc)/(ad) & b/d \\ 0 & 1\end{bmatrix}\begin{bmatrix}1 & 0 \\ c/a & 1\end{bmatrix}\begin{bmatrix} a/d & 0 \\ 0 & 1\end{bmatrix}d$$

Rescale by $1-(bc)/(ad)$ as the approach above to get

$$A = \begin{bmatrix} 1 & b/d \\ 0 & 1\end{bmatrix}\begin{bmatrix} 1-(bc)/(ad) & 0 \\ 0 & 1\end{bmatrix}\begin{bmatrix}1 & 0 \\ c/a & 1\end{bmatrix}\begin{bmatrix} a/d & 0 \\ 0 & 1\end{bmatrix}d$$

So now we have, from right to left, rescale, stretch, shear, stretch, shear. There might be other ways to write the same matrix using those basic transformations. Notice that I did not try to use the rotation.

Apply this to your example and it should work.