# Matrix Multiplication - Undefined Product

I am learning linear algebra for a machine learning class and have a question about matrix multiplication. The product of two matrices is undefined whenever the rows of the first matrix (reading right to left) do not match the column of the second matrix.

However, say that I would need (for some reason) to multiply a 3x3 matrix by a 2x2 one. Couldn't I just complete the operation by adding the "missing row" with coordinates [0,0]? I am thinking about it because if matrices represent linear transformations of space, then a 2x2 matrix represent a two dimensional transformation. However, isn't a two dimensional transformation simply a transformation where every other dimension is equal to 0?

To illustrate that, let's say I want to apply a linear transformation [-1,0;0,1] to vector [3,3]. The resulting vector would be a two dimensional vector [-3,3]. Now let's say that after this transformation I want to apply another transformation to the same vector, but this time in three dimensions. To keep the example as simple as possible let's use the identity transformation for this: [1,0,0;0,1,0;0,0,1]. Doing this would require to multiply the 3x3 matrix [1,0,0;0,1,0;0,0,1] by the 2x3 one [-1,0;0,1], which is technically not possible. However, if I apply the method above (i.e. adding a third row with all 0 to the 2x3 matrix) I would still be able to compute the transformation and get the result [-1,0,0;0,1,0]. I can then multiply this to my original vector and get [-3,3,0]. The only difference I can see between [-3,3,0] and [-3,3] is that in the first one I am just "explicitly showing" the third dimension as having coordinate 0, whilst in the second I am keeping this implicit.

What am I missing here?

Regards,

Federico

You need to be a little more explicit about what you want to do. Technically speaking, you simply cannot multiply a 2x2 matrix by a 3x3 one. So I'm not sure what you're after there.

But if $$A\in\mathcal{M}_{2\times 2}$$ and you want to find $$B\in\mathcal{M}_{3\times 3}$$ so that whenever $$(x_1,x_2,x_3)\in\mathbb{R}^3$$ we have $$A(x_1,x_2)^T$$ identical to the projection of $$B(x_1,x_2,x_3)^T$$ onto its first two coordinates, then that's definitely doable. Indeed, suppose $$A=\begin{pmatrix} a & b \\ c & d \end{pmatrix}$$ Then you need to let $$B=\begin{pmatrix} a & b & 0 \\ c & d & 0 \\ * & * & * \end{pmatrix}$$ where the stars can be anything you like.

• Hi @Ben W, I am not looking into any particular application. I am trying to understand conceptually why to use your words "Technically speaking, you simply cannot multiply a 2x2 matrix by a 3x3 one". – fcorte Aug 23 at 10:10
• @fcorte You already know that matrix multiplication is only defined for $\ell\times m$ and $m\times n$ matrices (i.e. where the number of columns of the first matrix matches the number of rows of the second). I'm not sure what more there is to say about that. – Ben W Aug 23 at 10:58
• Why is it only defined for l x m and m x n if one can still calculate a linear transformation by multiplying l x m with a n x n? That is, by just assuming any additional coordinate or basis vector in the higher dimensional matrix can be collapsed to 0 in the lower dimensional one when applying the standard multiplication rules. If you do this you still get a defined answer, albeit one the implies losing some info when collapsing from n+x dimensions into n ones. See also my question to Chris Culter below. – fcorte Aug 23 at 15:23
• @fcorte You can define the linear map $q:M_{2\times 2}\to M_{3\times 3}$ by letting $qA$ be the matrix formed by appending zeros to the left and bottom of $A$. Then for any $B\in M_{3\times 3}$ you can multiply $qA$ and $B$ together. That's fine as far as it goes, but it's not the same as multiplying $A$ and $B$, which operation is undefined. If you want, you can "extend" the definition of matrix multiplication and write $AB$ for $(qA)B$ when context is clear. But I'm not sure how useful that would be. – Ben W Aug 23 at 15:58

I'd say the core of the idea can be used, but a matrix containing only 0 doesn't leave the vector unchanged, the unit map does not change the vector, so if you want to use a 2D map on a 3D object you can opt to choose 1 axis along which nothing will be changed using the unit map rather than the zero map.

• Every square matrix represents a linear map regardless of its entries. So the zero matrix and the identity matrix are both linear, for example. – Ben W Aug 23 at 9:43
• Sorry, bad wording. A matrix containing only 0 is a linear map, but not one that does nothing. I tried changing it in my answer. – Emiel Lanckriet Aug 23 at 9:46

You're effectively doing calculations in $$\mathit{FSM}(\mathbb R)$$, the rng of finitely supported $$\mathbb N\times\mathbb N$$-dimensional real-valued matrices. And you're considering each such matrix to be equivalent to any finite-dimensional slice that contains all of its nonzero entries. This generally works fine: for example, multiplication in $$\mathit{FSM}(\mathbb R)$$ is associative. But there is no identity matrix, since it would need to have infinitely many $$1$$s, and that's inconvenient.

That said, just because you can do this doesn't necessarily mean you should! To phrase this in an applied setting: Let's say you're designing a linear algebra package from scratch. If the user tries to multiply two matrices with unexpected shapes, then it's likely a mistake, so it's better to signal a big fat error, rather than silently convert half of their data into zeroes. If the user really wants to perform the multiplication, require them to explicitly describe how they want the inputs to be beaten into shape first. For example, see numpy's array manipulation and pad functions.

• I think I get the point of your answer. To clarify: it seem to me there is no difference between matrices [a,b;c,d] and [a,b,n;c,d,n;n,n,n] for n=0. Is this what you also mean with "each such matrix to be equivalent to any finite-dimensional slice that contains all of its nonzero entries"? If correct and as my example above implies, then multiplying a 3x3 matrix by 2x2 one causes three dimensions to collapse into two. I see how this loses some "information", but isn't that still a "defined" case that in fact can be calculated using the standard rules? Why is this considered "undefined"? – fcorte Aug 23 at 15:13