# When reducing $A$ to $I$ via elementary matrices $E_k...E_1A=I$, why is it true also that $AE_k...E_1=I$

Say we have a $$3 \times 3$$ matrix $$A$$ and we perform the row operation $$Z_1:(R_1 \iff R_2)$$ of switching the first and second row to get $$A'$$.

Now, if we perform the same row operation $$Z_1:(R_1 \iff R_2)$$ on the identity matrix $$I_3$$ we get the elementary matrix $$E_1$$ and it is a common fact of linear algebra that $$E_1A = A'$$. Note that it is important that we multiply $$E_1$$ on the left of $$A$$.

Now, say further that $$A$$ is invertible and that we want to find $$A^{-1}$$. Then what we do is set up the augmented matrix $$[A | I_3]$$ and perform row operations $$Z_1,...,Z_k$$ until we we get an equivalent augmented matrix of the form $$[I_3|B]$$. It is a common fact of linear algebra that $$B = A^{-1}$$.

Now, the row operations $$Z_1,...,Z_k$$ correspond to the elementary matrcies $$E_1,...,E_k$$ and we can thus see that $$A^{-1}=E_k...E_1$$.

This is great and makes perfect sense! We reduced $$A$$ to $$I_3$$ via row operations corresponding to elementary matrices, so it is clear that $$A^{-1}A = E_k,...E_1A=I_3$$.

However, it is also true that $$AA^{-1}=I_3$$ and thus we have that $$AE_k,...E_1 = I_3$$. In the case when applying a string of elementary matrices to a matrix $$A$$ such that the string reduces $$A$$ to the identity, it is no longer important whether we apply the multiplication of the matrices on the right or left.

The bold font was meant to emphasize the phenomonon that I was hoping somebody could clarify. Understand that I am not asking why $$A^{-1}A = AA^{-1}$$ but rather seeking to clarify the more subtle observation of how the elementary matrices that embody the row operations can be applied to either side of $$A$$ in the case when they reduce $$A$$ to $$I$$.

Thank you.

• Did you try a simple $2\times 2$ example to see why this might be the case? You should be able to build up intuition from that. Feb 11, 2021 at 15:47

I'm going to work with a $$2\times 2$$ example so that it is explicit and clear in each step what is happening. Let $$A = \begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix}$$, then

$$\begin{pmatrix} \frac{1}{4} & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & -5 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & \frac{1}{11} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ \frac{1}{2} & 1 \end{pmatrix} \begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \tag{1}$$

## Multiplying on the left

Step by step (so we can compare with multiplying on the right shortly):

$$\begin{pmatrix} 1 & 0 \\ \frac{1}{2} & 1 \end{pmatrix} \begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} = \begin{pmatrix} 4 & 5 \\ 0 & \frac{11}{2} \end{pmatrix} \tag{2}$$

Rescaling,

$$\begin{pmatrix} 1 & 0 \\ 0 & \frac{2}{11} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ \frac{1}{2} & 1 \end{pmatrix} \begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} = \begin{pmatrix} 4 & 5 \\ 0 & 1 \end{pmatrix} \tag{3}$$

Then

$$\begin{pmatrix} 1 & -5 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & \frac{2}{11} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ \frac{1}{2} & 1 \end{pmatrix} \begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} = \begin{pmatrix} 4 & 0 \\ 0 & 1 \end{pmatrix} \tag{4}$$

and finally

$$\begin{pmatrix} \frac{1}{4} & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & -5 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & \frac{2}{11} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ \frac{1}{2} & 1 \end{pmatrix} \begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \tag{5}$$

Let's then work this the opposite way to see what the row operations above are doing when we multiply on the right instead.

## Multiplying on the right

$$\begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} \begin{pmatrix} \frac{1}{4} & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 5 \\ -\frac{1}{2} & 3 \end{pmatrix} \tag{6}$$

So in the first part, this gave us a unit pivot in the upper left and it does the same here (which makes sense). Note that it actually became a column operation on the first column though as opposed to a row operation on the first row when multiplied on the left. Of course that distinction doesn't matter when multiplying diagonal matrices as in $$(5)$$..

$$\begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} \begin{pmatrix} \frac{1}{4} & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & -5 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ -\frac{1}{2} & \frac{11}{2} \end{pmatrix} \tag{7}$$

So our matrix $$E_3$$ again eliminated the upper right element.

$$\begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} \begin{pmatrix} \frac{1}{4} & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & -5 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & \frac{2}{11} \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ -\frac{1}{2} & 1 \end{pmatrix} \tag{8}$$

This one again was a column operation and instead of scaling the bottom row, it scaled the rightmost column. Finally,

$$\begin{pmatrix} 4 & 5 \\ -2 & 3 \end{pmatrix} \begin{pmatrix} \frac{1}{4} & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & -5 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & \frac{2}{11} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ \frac{1}{2} & 1 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \tag{9}$$

So again $$E_1$$ eliminated the bottom left element.

As we saw above, our rescaling elementary matrices keep that behavior, it's just a matter of whether it's a row or a column rescaling depending on if it is multiplied on the left or on the right. And you can see easily that if you had to switch rows, the same logic would apply. So the question then is: what are the elimination elementary matrices doing?

Let's look at equations $$(4)$$ and $$(7)$$ more closely. In $$(3)$$, our previous operations had made our matrix into upper triangular form and we were starting to eliminate upwards and so we had a unit pivot in the bottom right before eliminating upwards. In step $$(6)$$, we rescaled the upper left element. However in step $$(7)$$, we didn't get a $$0$$ below, but rather to the right. So what that means is that the elimination matrices do not eliminate upwards when multiplied on the right but rather eliminate to the right. Similarly, eliminating downwards is actually the same as eliminating to the left when multiplying on the right which you can see when you compare equation $$(2)$$ to equation $$(9)$$. The pivot that was created in the bottom right was then used to eliminate the element to the left of it. (To know that it's right/left, pay close attention to what the matrix multiplication is doing and which elements are at play.)

So in short: row rescalings get turned into column rescalings, row swaps get turned into column swaps, and eliminating downward (upward) gets turned into eliminating leftward (rightward).

• I went through several iterations of my matrix $A$ because at first I had two $1$s, then realized things seemed too "easy" and may have been a trick, then I had one $1$, but same issue there. Settled on this to make it clear just how the cancellation all works out. Feb 11, 2021 at 17:13
• doesn't you're line corresponding to equation $(1)$ have a flaw in it? i.e. the first elementary matrix is not an elementary matrix and the matrix that you are reducing to the identity is not the same as originally planned Feb 12, 2021 at 1:48
• You're right about the first one not being an elementary matrix. I always forget to split this into two steps. And let me fix that typo. Give me a few and it'll be fixed throughout. Feb 12, 2021 at 13:36
• Wow good grief I had a bunch of typos. I guess that's what happens when you change your example several times throughout. Feb 12, 2021 at 13:46

If $$E_2E_1A=I\tag{1},$$ then $$A=E_1^{-1}E_2^{-1}$$ and thus $$AE_2E_1=(E_1^{-1}E_2^{-1})E_2E_1=I\tag{2}$$

This simply says that if you can transform $$A$$ into the identity matrix by pure row operations, you can also do it by pure column operations.

But the orders of the operations you do are different.

In (1), you first do the row operation $$E_1$$ and then the operation $$E_2$$. But in (2), you first do the column operation $$E_2$$ and then the operation $$E_1$$.

it is no longer important whether we apply the multiplication of the matrices on the right or left

If this were true, what you should expect is $$E_2E_1A=AE_1E_2$$ which is in general not true.

• OP already says they understand this part. What they don't understand is why at a row/column operations level this should be the case. Feb 11, 2021 at 15:58
• The point is that they are not the same.
– user9464
Feb 11, 2021 at 15:59
• Thanks. Although I don't feel completely satisfied with this answer, viewing $A$ as such did help a bit. But still, I think Cameron above has found a better way to phrase my question as why this is the case at the row/column operation level. Feb 11, 2021 at 16:00