# Question regarding the intuitive, visual interpretation of matrix multiplication

I have a question on one of 3Blue1Brown's wonderful youtube videos regarding what matrix multiplication visually is.

It can be found here.

The part of the video I'm concerned with starts at roughly $6:20$. What I gathered, is that the narrator says that any matrix $A$ can be pictured as a transformation of basis vectors $\hat i + \hat j$ into $A$ by $IA$, where $\hat i + \hat j$ is the identity matrix and $A$ is what the basis vectors \begin{bmatrix} 1 & 0 \\ 0 & 1\\ \end{bmatrix}will now orient themselves as.

When two matrices, $C$ and $D$ are multiplied together, I interpreted this as essentially taking $I$, multiplying it by $D$, then taking that transformed basis vector and multiplying it by $C$. And then, we have a new basis vector with a linearly transformed space, and any vector, like $2\hat i + 3\hat j$ can be translated into this new space by multiplying the transformed basis vectors by $2$ and $3$ respectively.

If this is all true, then do we, when performing matrix multiplication, interpret $CD$ as basis vector matrix $D$ is transformed by $C$ to form the new basis vector orientation $CD$, which is a matrix that changes the space around it is a basis for evaluating the projection of other vectors?

For instance, let's say after multiplication I have the following matrix: \begin{bmatrix} 4 & -2 \\ 3 & 1\\ \end{bmatrix}

Does this, according to the video, mean our basis vector $\hat i$ is now $4\hat i + 3\hat j$? If so, $\hat i$ is now made up of projections $\hat i$ and $\hat j$? That seems a bit odd to me, like saying $x = 3x + y$. What bit am I not interpreting correctly?

Think of a matrix as a function that maps a vector to a new vector. (it follows certain rules of linear maps, but that is not entirely relevant right now). Matrix multiplication is a composition of two functions.

You can look at what matrices $AB$ does to your principle component vectors. Remmber composition works right to left. So $B$ takes those vectors somewhere. And $A$ takes them somewhere else.

suppose $A = \begin{bmatrix} 1&3\\-1&2 \end{bmatrix}$

Applied to the principle component vectors.

$A\begin{bmatrix} 1\\0 \end {bmatrix} = \begin{bmatrix} 1\\-1 \end{bmatrix}\\ A\begin{bmatrix} 0\\1 \end {bmatrix} = \begin{bmatrix} 3\\2 \end{bmatrix}$

And applied to an generalized vector.

$A\begin{bmatrix} x_1\\x_2 \end {bmatrix} = \begin{bmatrix} 1\\-1 \end{bmatrix} x_1 + \begin{bmatrix} 3\\2 \end{bmatrix} x_2$

Let $B = \begin{bmatrix} -1&2\\1&1 \end{bmatrix}$

$AB\begin{bmatrix} 1\\0 \end {bmatrix} = A\begin{bmatrix} -1\\1 \end{bmatrix}=\begin{bmatrix} 1\\-1 \end{bmatrix} (-1) + \begin{bmatrix} 3\\2 \end{bmatrix} (1)=\begin{bmatrix} 2\\3 \end{bmatrix}\\ AB\begin{bmatrix} 0\\1 \end {bmatrix} = A\begin{bmatrix} 2\\1 \end{bmatrix}=\begin{bmatrix} 1\\-1 \end{bmatrix} (2) + \begin{bmatrix} 3\\2 \end{bmatrix} (1)= \begin{bmatrix} 5\\0 \end{bmatrix}\\ AB\begin{bmatrix} x_1\\x_2 \end {bmatrix} = \begin{bmatrix} 2\\3 \end{bmatrix}x_1 + \begin{bmatrix} 5\\0 \end{bmatrix} x_2$

I think one source of your confusion is that there are a few different geometric interpretations for a matrix.

First, let me comment on Doug M's lovely answer. He consistently uses the interpretation of an $m$ by $n$ matrix $A$ as the linear transformation $\textbf{x} \mapsto A\textbf{x}$, where $\textbf{x} \in \mathbb{R}^n$ and $A\textbf{x} \in \mathbb{R}^m$. That is the best (but not only) interpretation of a matrix when you are trying to understand matrix multiplication in a fundamental way, since, as stated somewhere, matrix multiplication represents the composition of two such linear transformations. To be very pedantic, let $S: \mathbb{R}^n \to \mathbb{R}^m$ be defined by $T(\textbf{x}) = B\textbf{x}$. (So $A$ is $m$ by $n$ and $B$ is $n$ by $p$.) Then the composed function $S \circ T$ is represented by the matrix product $AB$, that is,$$(S \circ T)(\textbf{x}) = (AB)(\textbf{x}).\tag*{(*)}$$(Note that this can be seen as a special case of the associativity of matrix multiplication: it says that$$(AB)(\textbf{x}) = S(T(\textbf{x})) = A(B\textbf{x})$$but it is actually much easier to prove $(*)$ first and then show the matrix multiplication must be associative because function composition is.) Also, the example in his answer is a good model of how the proof of $(*)$ goes in general. So that is all good.

Now back to what you wrote. You say the following.

$\ldots A$ can be pictured as a transformation of basis vectors$\ldots$

But it is rather vague what you mean by that. Also, you wrote the following.

$\widehat{i} + \widehat{j}$

I think you really mean "$\widehat{i}$ and $\widehat{j}$", an ordered set of two vectors (in fact a basis), not just the one vector that is the sum of $\widehat{i}$ and $\widehat{j}$. Similarly, the rest is a bit unclear to me.

So let me give you an interpretation that seems to be close to what is being said. The key is that (unlike above) we will use different interpretations of what the two matrices in a product represent. So suppose that we have (for simplicity) two square (say $n$ by $n$) matrices $C$, $D$. The matrix $D$ we interpret as an ordered collection of column vectors, put together for notational convenience (although we will see that we get more out of this trick in a minute). In nice cases, these column vectors span $\mathbb{R}^n$ and are linearly independent (in fact they do one job exactly when they do the other, and also exactly when $D$ is invertible—that is a big theorem), i.e. they form a basis of $\mathbb{R}^n$, but it is not essential for this argument. The matrix $C$, on the other hand, we interpret as encoding a linear transformation, just as before. Then we have the basic definition of matrix multiplication, namely$$\text{the }k\text{the column of }CD := C(\text{the }k\text{th column of }D)$$where the product on the right is the usual matrix times vector multiplication. (This may not be the definition you have seen of matrix multiplication, but it is the best one to start with, and is equivalent to all of the other characterizations. it is also the definition that makes the proof of $(*)$ almost tautological.) So what is happening here? We are just applying $C$ (which is a transformation) to each vector in the ordered set of vectors encoded by $D$. Hence the columns of $CD$ can be interpreted as a new ordered set of vectors, namely the vectors from $D$ transformed by $C$. Very natural. In nice cases, not only will the columns of $D$ form a basis, but also $C$ will be invertible, in which the columns of $CD$ will form a basis of $\mathbb{R}^n$ as well.

One example (which I think is alluded to) is the case where $D =1$ is the identity matrix, i.e. where the basis we start with is the standard basis of $\mathbb{R}^n$. Then what we are doing is creating a new set of vectors (a new basis, if $C$ is invertible). We are not saying that this new basis is the same as the old basis—it has been purposely transformed. So in your example, where$$C = \begin{pmatrix} 4 & -2 \\ 3 & 1 \end{pmatrix}$$the new basis is the set of vectors $\{4\widehat{i} + 3\widehat{j}, -2\widehat{i} + \widehat{j}\}$. We could then use that basis to express another vector $\textbf{v}$ as a combination of those two vectors, for example, or do the other things bases are good for. But it is important that this is a truly new basis.

More generally, we could have started with a nonstandard basis, say $\{5\widehat{i} - \widehat{j}, 6\widehat{i} + \widehat{j}\}$, and we might want to see what happens when we transform these vectors by the transformation $\textbf{x} \mapsto C\textbf{x}$ where$$C = \begin{pmatrix} 2 & -1 \\ 7 & -3\end{pmatrix}.$$To encode this we can create$$D = \begin{pmatrix} 5 & 6 \\ -1 & 1\end{pmatrix}$$and calculate$$CD = \begin{pmatrix} 2 & -1 \\ 7 & -3\end{pmatrix}\begin{pmatrix} 5 & 6 \\ -1 & 1\end{pmatrix} = \begin{pmatrix} 11 & 11 \\ 38 & 39 \end{pmatrix}$$which tells us that after transforming, the two basis vectors have now become two new vectors $\{11\widehat{i} + 38\widehat{j}, 11\widehat{i} + 39\widehat{j}\}$. (It is interesting that these happen to be rather close to being parallel—that was unplanned. Note though that since $C$, $D$ are both invertible in this example, these two vectors are guaranteed not to be exactly parallel.)

There are various other interpretations for a matrix, but these two (especially the one of Doug M's answer, the matrix as a linear transformation) are a good start in thinking about the geometry of matrix multiplication. Hope this helps—one thing that is important is to draw a lot of pictures in 2D of what bases and transformations can look like, and I have not done that here, sorry.