22
$\begingroup$

Most of the time I see matrix multiplication presented and defined, as a seemingly arbitrary sequence of operations. For example, the textbook I'm currently reading for a linear algebra course defines the product $AB$ as the $(i, j)$ entry in the $1 \times 1$ matrix that is the product of the ith row of $A$ and the jth column of $B$. Properties of matrix multiplication are subsequently proven based upon this definition. The definition is clear, but why the matrix product is useful is not clear to me as a student. A different textbook I'm referencing defines the product $AB$ in terms of linear combinations.

The problem I have is doing matrix multiplication quickly by hand, particularly when the $A$ is $p \times 1$ and $B$ is $1 \times q$. I would like to know of how to look at or define matrix multiplication, in a manner which makes it easy (for the average student) to compute by hand, while being intuitive and consistent for use for later proofs.

wikipedia has a great article.

$\endgroup$
  • 1
    $\begingroup$ See also my answer to a similar question $\endgroup$ – Ben Millwood Sep 8 '12 at 18:52
  • $\begingroup$ $A= \left( \begin{matrix} a_{11}&a_{12}&a_{13} \\a_{21}&a_{22}&a_{23} \\a_{31}&a_{32}&a_{33} \end{matrix} \right)$ $\endgroup$ – T. Webster Apr 27 '13 at 4:53
7
$\begingroup$

In general, let $A$ be $m\times n$ and let $B$ be $n\times l$. Let $A_i=(a_{i1},a_{i2},\dots,a_{in})$ be the $i$th row of $A$, $1\leq i\leq m$ and let $B_j=(b_{1j},b_{2j},\dots,b_{nj})$ be the $j$th column of $B$, $1\leq j \leq l$. Given a row vector $\overline x=(x_1,x_2,\dots,x_n)$ and a row vector $\overline y =(y_1;y_2;\dots;y_n)$, define $$\overline x\cdot \overline y=\sum_{i=1}^n x_iy_i$$

Then we can define $$A\cdot B$$ as the matrix who has as entries $$(AB)_{ij}=A_iB_j=\sum_{k=1}^n a_{ki}b_{jk}$$

where we interpret each row and column as a row vector and column vector respectively.

You can write it as $$A\cdot B=\left( \begin{matrix} A_1B_1&A_1B_2&\cdots&A_1B_l\\ A_2B_1&A_2B_2&\cdots&A_2B_l \\\vdots&{}&{}&\vdots \\A_mB_1&A_mB_2&\dots&A_mB_l\end{matrix} \right)$$

$\endgroup$
39
$\begingroup$

Perhaps the best way to look at matrix multiplication when you want to compute a product by hand is as follows:

matrix multiplication

(See also the comparable illustration on the wikipedia page.)

When you're computing by hand, just bump the second matrix up to make room for the product in the lower-right corner.

Note that the upper-left corner must be a square, which re-confirms the requirement that "columns of $A$" = "rows of $B$"; moreover, you can see that $A\times B$ inherits its row dimension from $A$ and its column dimension from $B$.

$\endgroup$
23
$\begingroup$

Matrix multiplication is defined as it is so that is reflects the composition of linear maps. No more, no less.

$\endgroup$
  • 5
    $\begingroup$ This is the reason why matrix multiplication is defined this way. $\endgroup$ – F M Sep 14 '12 at 4:06
9
$\begingroup$

It's important to understand how to multiply $AB$ by recognizing that each column of $AB$ is a linear combination of columns of $A$ with the corresponding column of $B$ telling you which particular linear combination; similarly each row of $AB$ is a linear combination of the rows of $B$ with the corresponding row of $A$ telling you which particular linear combination.

This is covered in any reasonable text on linear algebra. This perspective is both helpful for doing concrete calculations by hand as well as for understanding matrices theoretically. In particular, this interpretation of matrix multiplication is very handy for understanding Gaussian elimination and for studying the rank of a matrix.

$\endgroup$
  • 3
    $\begingroup$ Thats my favorite method. The great thing is that when the numbers are reasonable, I am able to compute a whole row at "once" instead of advancing by single numbers. $\endgroup$ – Honza Brabec Sep 8 '12 at 20:18
9
$\begingroup$

There is already a really great answer on why matrix multiplication is defined as it is, so this shall be the only mention of it in this answer. Instead I will show you, how I normally handle these multiplications, and why this particluar way of doing them is better suited for hand- or head-calculations then others. $\newcommand{\vek}[1]{\boldsymbol{#1}}$

Before I really start, I will swiftly go over some notation, and before you really get your answer, I will explain how I do matrix-vector multiplications, since this will be used in for the matrix-matrix multiplication.

Notation

Let $A \in K^{q \times n}$ and $B \in K^{n \times p}$ be two matrices and $\boldsymbol{\nu} \in K^n$ a vector, where $K$ is any field that tickles your fancy. I will write the vectors in bold, to differentiate them more clearly form numbers in a matrix.

Now for $A$ we can write $$ A = \begin{pmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{q1} & a_{q2} & \dots & a_{qn} \\ \end{pmatrix} = \begin{pmatrix} & & & \\ \boldsymbol{a}_{1} & \boldsymbol{a}_{2} & \dots & \boldsymbol{a}_{n} \\ & & & \\ \end{pmatrix} $$ where we defined the column-vectors $\boldsymbol{a}_{j}~$ . To be precise, we have $$ \boldsymbol{a}_{j} = \begin{pmatrix} a_{1j} \\ \vdots \\a_{qj} \end{pmatrix} $$ And we do the same thing in $B$ to get column vectors $\boldsymbol{b}_{j}$


Matrix-Vector mutliplication

Let's say you want to evaluate $A \boldsymbol{\nu} = \boldsymbol{\omega}$ then you will normally find something like $$ \sum_{j=1}^n a_{ij} \nu_j = \omega_i ~~~~~~\text{for every} ~~~~i \in \{1,\dots,q\} \tag{1} $$ as one possible way to get $\vek{\omega}$. So practically you calculate every component of $ \boldsymbol{\omega}$ seperately. Instead of this I always do it, and I suggest you do as well, in the following way

$$ A \boldsymbol{\nu} = \begin{pmatrix} & & & \\ \boldsymbol{a}_{1} & \boldsymbol{a}_{2} & \dots & \boldsymbol{a}_{n} \\ & & & \\ \end{pmatrix} \begin{pmatrix} \nu_{1} \\ \vdots \\ \nu_{n} \end{pmatrix} = \nu_1 \boldsymbol{a}_{1} + \nu_2 \boldsymbol{a}_{2} + \dots + \nu_n \boldsymbol{a}_{n} = \boldsymbol{\omega} \tag{2} $$ Here, you multiply every column vector $\vek{a}_j$ of $A$ by $\nu_j$, and add those resulting vectors up.

So what is the difference?

$(1)$ If you do the calculation component-wise, then you calculate $\omega_1$ by taking $a_{11}$ and multiplying it by $\nu_1$ to which you add $a_{12}$ multiplied by $\nu_2$ ... Then you calculate $\omega_2$ by taking $\nu_1 a_{21}$, adding $\nu_2 a_{22}$ ... Until you have calculated $\omega_q$, at which point you are done. So in every step (i.e. for every $\omega_i$), you have to go back to your vector $\boldsymbol{\nu}$, check again what $\nu_1$ was (because you have propably forgotten it when you got to $\nu_3$), look at what $a_{i1}$ is, and do the multiplication.

$(2)$ Using the column vectors of $A$, you check the value of $\nu_1$ once, multiply every component of $\boldsymbol{a}_1$ by it, and write down the resulting vector. Now you never have to think about $\nu_1$ ever again, you just continue with $\nu_2$, $\nu_3$ etc. And when all the $\nu_j \boldsymbol{a}_j$ lay written down before you, you may just squash (add) them together to get the desired result.

So by using $(2)$ you avoid a lot of going back to look at what $\nu_j $ was.

Matrix-Matrix multiplication

Now we get to the bit you asked for. If I have to calculate $AB$, I use the following fact

$$ AB = A \begin{pmatrix} & & & \\ \boldsymbol{b}_{1} & \boldsymbol{b}_{2} & \dots & \boldsymbol{b}_{p} \\ & & & \\ \end{pmatrix} = \begin{pmatrix} & & & \\ A \boldsymbol{b}_{1} & A \boldsymbol{b}_{2} & \dots & A \boldsymbol{b}_{p} \\ & & & \\ \end{pmatrix} \tag{3} $$ using $(2)$ to calculate the $A \boldsymbol{b}_{j}$'s.

Let me convince you that the above is actually true, so that you can use it without concern.

We want to know what the $j$-th column vector of $AB$ is. Now if we take $\boldsymbol{e}_j$ to be the vector which has 0 everywhere, except for the $j$-th component, which is 1, we have, by $(2)$, that $(AB) \boldsymbol{e}_j$ is the column vector we are looking for. But this also equates to $$ (AB) \boldsymbol{e}_j = A(B \boldsymbol{e}_j) = A \boldsymbol{b}_j $$ Again, the advantage of calculating the product like this is that you have to remember some values less often. This time you will have to check some values several times. But it's still worse if you use the $(AB)_{ij} = \sum_k a_{ik} b_{kj}$ -Way to do the multiplication. There are some other cases, where handling matrix multiplication in this way can be useful.


Usages

  • Let $A$ be a diagonalizable matrix with Eigenvectors $s_j$ who have Eigenvalues $\sigma_j$, $D$ its diagonal matrix, and $S$ the transformation matrix. What I could always remember was that $S$ contains the Eigenvectors of $A$, and that either $ S^{-1} A S = D$ or $ S A S^{-1} = D$, but I could never just remember which way. Using $(3)$ to carry out the multiplications you get $$ S^{-1} A S = S^{-1} A \begin{pmatrix} \\ \boldsymbol{s}_{1} \dots \\ \\ \end{pmatrix} = S^{-1} \begin{pmatrix} \\ A \boldsymbol{s}_{1} \dots \\ \\ \end{pmatrix} = S^{-1} \begin{pmatrix} \\ \sigma_1 \boldsymbol{s}_{1} \dots \\ \\ \end{pmatrix} = \begin{pmatrix} \\ \sigma_1 S^{-1} \boldsymbol{s}_{1} \dots \\ \\ \end{pmatrix} = \begin{pmatrix} \\ \sigma_1 \boldsymbol{e}_{1} \dots \\ \\ \end{pmatrix} = D $$
  • If you want to evaluate a vector-matrix product, i.e. something like $\vek{\nu}^T A = \vek{\omega}$, where $\vek{\nu}$ is a column vector, then there is already everything you need to discuss this case as well, since $$ \vek{\omega}^T = (\vek{\nu}^T A)^T = A^T \vek{\nu} $$ where you can use $(2)$ , and to get $\vek{\omega}$ you just do the transpose again. If you carry out the calculation, you will see that this leads to the general result $$ \vek{\nu}^T A = \begin{pmatrix} \nu_1 & \dots & \nu_q \end{pmatrix} \begin{pmatrix} & \vek{a}_1 & \\ & \vdots & \\ & \vek{a}_q & \end{pmatrix} = \nu_1 \vek{a}_1 + \dots + \nu_q \vek{a}_q = \vek{\omega} $$ Where the $\vek{a}_j$'s are now the row vectors of $A$.

  • In some cases, when the matrix A doesn't look too scary, you can just look at the matrix to find it's inverse (if invertible of course). Just ask yourself: what does the $j$-th column vector of the inverse has to accomplish? It gives you a linear combination of the column vectors of $A$, which have to add up to $\vek{e}_j$ . So if you can see what linear combination does the job, you get one column of $A^{-1}$ .
    Here an easy example: $$ A = \begin{pmatrix} 0 & -1 \\ 1 & 2 \end{pmatrix} $$ To get $\vek{e}_1$ you take $ 2 \cdot \begin{pmatrix} 0 \\ 1 \end{pmatrix} -1 \cdot \begin{pmatrix} -1 \\ 2 \end{pmatrix} ~~$ and $~~\vek{e_2} = 1 \cdot \begin{pmatrix} 0 \\ 1 \end{pmatrix} + 0 \cdot \begin{pmatrix} -1 \\ 2 \end{pmatrix} $
    So the inverse is $$ A^{-1} = \begin{pmatrix} 2 & 1 \\ -1 & 0 \end{pmatrix} $$

$\endgroup$
  • 1
    $\begingroup$ an excellent answer. Especially the matrix matrix part that lends itself to the student of eigenvectors and values. $\endgroup$ – Karl Aug 6 '16 at 12:20
6
$\begingroup$

Just remember that each ij-th entry in AB (where i signifies the row of AB and j signifies the column in AB) is equal to the the dot product of the ith row of A and the jth row of B.

Another trick is to visualize turning the jth column of B and aligning it with the ith row of A, multiplying each entry, and then adding it up.

$\endgroup$
  • $\begingroup$ That is literally the most important thing to bear in mind, thanks! $\endgroup$ – Joel Apr 27 '19 at 14:02
0
$\begingroup$

To expand on the answer by user39794:

The dot product can be thought of intuitively as the vector A in the direction of vector B.

An example use of the dot product could be the amount of light bouncing off an object in the direction of your eye.

As Matrix multiplication produces a matrix comprising the dot product of all rows in matrix A with all columns in matrix B, it could be thought intuitively as matrix A in the direction of matrix B.

An example use of matrix multiplication could be a 3D model in the direction of the camera.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.