Why multiplication of matrix is not done in the same way as matrix addition (i.e. adding corresponding entries)? I know it is related to linear transformation, but by reading book I'm unable to visualize it.

  • $\begingroup$ Matrix-vector multiplication is actually just shorthand notation for a linear map. If you were to compose two linear maps and then separate them as a product, the usual multiplication as matrices is exactly what pops out. There's nothing stopping you from defining a new multiplication, but it might not be super useful. $\endgroup$ – Cameron Williams Dec 28 '17 at 4:01
  • 2
    $\begingroup$ You can define matrix multiplication any way you like, but not all ways are equally useful in mathematics. The usual way of multiplying turns out to be the most useful. If defined the way you suggest, $m\times n $ matrices over a field $F $ would just turn into a commutative ring $F^{mn}$ (which is a legitimate, though not very interesting, object to study; for example, it has zero divisors). $\endgroup$ – user491874 Dec 28 '17 at 4:06
  • $\begingroup$ @user8734617: I assume you know that there are hundreds of research papers on the Schur/Hadamard product? $\endgroup$ – Martin Argerami Dec 28 '17 at 13:09

Suppose \begin{align} p & = 3x+10y, \\ q & = 7x-2y, \end{align} and \begin{align} a & = -13p + 9 q, \\ b & = \phantom{+}6p + 5q. \end{align} Then \begin{align} a &= 24 x -148y, \\ b & = 53x + 50y. \end{align} That's why matrix multiplication is defined the way it is.

  • 5
    $\begingroup$ +1 for simplicity. Definitions are made to be convenient. $\endgroup$ – Myridium Dec 28 '17 at 9:22
  • $\begingroup$ I feel that your two sets of equations would be better the other way around. As it is if you transform them to matrices for multiplication then the second set is the left hand matrix and the first set is the right hand matrix in the multiplication (I think). $\endgroup$ – Chris Dec 28 '17 at 10:51
  • 1
    $\begingroup$ I think this would greatly benefit from actually writing it in matrix form. I.e: (p, q) = A * (x, y), (a, b) = B * (p, q) => (a, b) = B * A * (x, y) = C * (x, y) and C = B * A defines matrix multiplication. $\endgroup$ – WorldSEnder Dec 28 '17 at 17:54
  • $\begingroup$ @Chris : Being the left-hand element means it is done after the right-hand element. $\endgroup$ – Michael Hardy Dec 28 '17 at 18:56
  • $\begingroup$ Hmmm... Yes, the way WorldSEnder describes it is what I was thinking. In terms of the matrices you have B * A = C whereas I would think having A * B = C would make more sense. My immediate thought to understand what you did was to multiply the first two sets of equations as matrices (admittedly without thinking too hard as whether what I was doing made sense) and I got the wrong answer which confused me for a while. $\endgroup$ – Chris Dec 29 '17 at 9:12

In short, matrix multiplication is defined the way it is because it corresponds to the composition of linear transformations on (finite dimensional) vector spaces. This is what Cameron Williams mentioned in the comments.

For a very simple example of this in action, consider rotations of vectors in the plane $\mathbb{R}^2$. A rotation is an example of a linear transformation, which can be represented as a matrix. For instance, rotation counterclockwise by $90^\circ$ can be represented by the matrix

$$ R_{90} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} $$

Note that if you start with the vector $(1,0)^T$ then $R_{90}$ rotates it by $90^\circ$ as follows

$$ \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 0 \\ 1 \end{pmatrix} $$

In the same way if you do the calculation with any other vector in $\mathbb{R}^2$, the matrix $R_{90}$ will just rotate the vector by $90^\circ$. You can obtain a matrix like this for any degree of rotation. For example

$$ R_{180} = \begin{pmatrix} -1 & 0 \\ 0 & -1 \end{pmatrix} $$

is the matrix corresponding to rotation counterclockwise by $180^\circ$. Try it out on a few vectors if you haven't seen this before.

Now to the point. Note that rotations may be composed (i.e. we may perform one rotation after the other to achieve a total rotation). Composing two $90^\circ$ rotations results in an overall $180^\circ$ rotation. In the same way that we can naturally compose these linear transformations it would be nice if there was a way to "compose" the matrix $R_{90}$ with itself to produce the matrix $R_{180}$. Well, because of the way matrix multiplication is defined, we can do just this by multiplying $R_{90}$ by itself. Observe that

$$ R_{90}\cdot R_{90} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} = \begin{pmatrix} -1 & 0 \\ 0 & -1 \end{pmatrix} = R_{180} $$

  • 2
    $\begingroup$ This is a good answer. Simple examples are the foundations of understanding a new concept. The OP should note that this “resemblance” of (orthogonal) matrix multiplication to rotation composition can be shown more abstractly to hold for general linear transformations. A good reference to picturing this is 3Blue2Brown’s video on the topic, which can be found on YouTube. $\endgroup$ – giobrach Dec 28 '17 at 13:00

Matrices are sometimes multiplied in this way. This is known as Hadamard product, Schur product, or entrywise product, and comes up in the theory of association schemes, for example.

Matrix multiplication is often taught as a completely arbitrary operation. This is similar to defining multiplication of integers by specifying the long multiplication algorithm. It is much better to think of matrix multiplication semantically. There are at least two prominent semantic ways to think about matrices:

  1. A matrix is a way of representing a linear transformation given a specific basis for the underlying vector space. When you compose two linear transformations, the matrix of the composed transformation is obtained by matrix multiplication.

  2. A system of linear equations can be expressed in the form $Ax = b$. When $x$ and $b$ have the same dimensions and there is a unique solution, the unique solution is given by the formula $x = A^{-1}b$, where $A^{-1}$ is the multiplicative inverse of $A$ with respect to matrix multiplication.

There are other situations in which matrix multiplication natively occurs, but these are perhaps the two simplest ones. Also, it's not really a coincidence that the same notion of matrix multiplication works in both situations — we can think of a system of linear equations as a specification for a linear transformation, and we use this insight to solve the system.

  • $\begingroup$ I guess the question still remains why we choose to denote composition of linear operators as multiplication in the matrix world. We could instead have denoted it as "composition of matrices". But I suspect the reason is that the most common ring of matrices is the one given by defining matrix multiplication that way, and the second ring operation is typically denoted as multiplication. $\endgroup$ – Reinstate Monica Dec 28 '17 at 18:29

A matrix is usually not seen as just a bunch of numbers arranged in a rectangular pattern. In many cases a matrix is seen as something that transforms a vector. You then combine vector transformations, and figure out what numbers you would have to put into a matrix to produce that transformation.

A very common combination of transformations is "apply transformation 1 on the original vector, then apply transformation 2 on the result of the first transformation. ". The common matrix multiplication produces exactly the numbers that you would need to do this in a single transformation.

Another common combination is "apply transformation 1 on the original vector, apply transformation 2 on the original vector, then add the results". Elementwise addition produces a matrix representing this combination.

The situation where you want "apply transformation 1 on the original vector, apply transformation 2 on the original vector, then multiply the results" seems uncommon.

  • $\begingroup$ If you multiply $Ax$ and $Bx$ entrywise then you won't get a linear transformation at all. Try it with $A=B=I$ (the identity). $\endgroup$ – Yuval Filmus Dec 28 '17 at 19:03

Not the answer you're looking for? Browse other questions tagged or ask your own question.