23
$\begingroup$

I am learning linear algebra, and I am a bit confused by the dot product and how the answer to the process turns out to be a scalar rather than a matrix.

For $2$ vectors with $2$ components, I learned that dot product is equivalent to a $1 \times 2$ row vector left multiplied by a $2 \times 1$ column vector. The result of such a multiplication should result in a $ 1 \times 1 $ vector. But I am learning that the dot product somehow transforms the result into a scalar, rather than a $ 1 \times 1 $ vector.

Maybe I am missing something but the difference between a $ 1 \times 1 $ vector and a scalar seems important, because you can multiply a scalar by a matrix of any size, but you can only left-multiply a $ 1 \times 1 $ vector by another matrix with $1$ row.

Thanks for any help in understanding this.

$\endgroup$
  • 2
    $\begingroup$ I guess what I am confused about is that I learned that matrix multiplication is only defined when the number of columns on the left matrix = the number of rows in the right matrix. In which case, a 1x1 matrix could only be left multiplied by other 1 row matrices, whereas a scalar can multiply by a matrix of any size. So simply converting a 1x1 matrix to a scalar seems to removes some of the restrictions that matrix multiplication has. $\endgroup$ – treeorriffic Mar 1 '17 at 17:06
  • 4
    $\begingroup$ You're not wrong, of course. But sometimes mathematicians take shortcuts. See my answer below for more. $\endgroup$ – Bobbie D Mar 1 '17 at 17:11
  • 1
    $\begingroup$ You might find my answer here useful $\endgroup$ – Omnomnomnom Mar 1 '17 at 17:13
  • 1
    $\begingroup$ If you want to replace scalars with $1 \times 1$ matrices, you need to replace the scalar product with the tensor product (i.e. kronecker product). i.e. $aM = [a] \otimes M = M \otimes [a]$ for any scalar $a$ and matrix $M$. Of course, if $M$ is $1 \times n$ then the matrix product works too: $[a]M = [a] \otimes M$. Similarly, $M[a] = M \otimes [a]$ if $M$ is $n \times 1$. $\endgroup$ – user14972 Mar 2 '17 at 1:23
  • $\begingroup$ A related issue is: is a vector of length $n$ the same as a $n\times 1$ matrix? $\endgroup$ – Federico Poloni Mar 2 '17 at 9:39
18
$\begingroup$

Sometimes, we just say that a $1\times 1$ matrix is the same as a scalar. Afterall, when it comes to addition and multiplication of $1\times 1$ matrices vs addition and multiplication of scalars, the only difference between something like $\begin{bmatrix}3\end{bmatrix}$ and $3$ is some brackets. Consider $$(3+5)\cdot 4 = 32 \\ (\begin{bmatrix} 3\end{bmatrix} + \begin{bmatrix} 5\end{bmatrix})\begin{bmatrix} 4\end{bmatrix} = \begin{bmatrix} 32\end{bmatrix}$$ The algebra works out exactly the same. So sometimes it's not ridiculous to think of $1\times 1$ matrices as just another way of writing scalars.

But if you do want to distinguish the two, then just think of the formula $a\cdot b = a^Tb$ as a way of finding out which scalar you get from the dot product of $a$ and $b$ and not literally the dot product value itself (which should be scalar). That is, we calculate the dot product of $\begin{bmatrix} 1 \\ 2\end{bmatrix}$ and $\begin{bmatrix} 3 \\ 4\end{bmatrix}$ by using the formula $$\begin{bmatrix} 1 \\ 2\end{bmatrix}^T\begin{bmatrix} 3 \\ 4\end{bmatrix} = \begin{bmatrix} 1 & 2\end{bmatrix}\begin{bmatrix} 3 \\ 4\end{bmatrix} = \begin{bmatrix} 11\end{bmatrix}$$ and then say that this tells us that the dot product is really $11$. So the formula $a^Tb$ is just an algorithm we use to find the correct scalar.

You can view it either way. It doesn't really make a difference.

$\endgroup$
  • 3
    $\begingroup$ Oh, viewing it as an algorithm, rather than the literal result, is really helpful. Thanks. $\endgroup$ – treeorriffic Mar 1 '17 at 17:17
11
$\begingroup$

You are not wrong and it's always good to examine statements very carefully. We usually do not distinguish 1$\times$1 matrices from scalars, and in some sense you can think of scalar multiplication as a special rule for when one of the matrices is 1$\times$1. But the truth is that it is a convenient abuse of notation.

Here's one other thing to think about. A $1 \times 1$ real matrix is supposed to represent a linear mapping from a one-dimensional real vector space into a one-dimensional real vector space--essentially just $\mathbb R$ into $\mathbb R$. The only such mappings are those that take $x \mapsto ax$ for some fixed real number $a$. But this mapping is entirely determined by that real number $a$, so they are essentially equivalent.

$\endgroup$
  • $\begingroup$ Ah thanks, that makes sense. It's just a special rule for 1x1 matrices. $\endgroup$ – treeorriffic Mar 1 '17 at 17:09
5
$\begingroup$

It is true that you can only multiply a $m \times n$ matrix by a $n \times p$ matrix, i.e., the column size of the left matrix has to match the row size of the right matrix. With this, we can conclude that a product of a $1 \times 1$ matrix by a $n \times p$ matrix makes no sense for $n > 1$.

That being said, the space of matrices is a vector space, so it has the multiplication between scalars and matrices. So it makes sense to multiply a scalar by a matrix. Once you realize the space of scalars and $1 \times 1$ matrices can be identified, the confusion goes away.

$\endgroup$
5
$\begingroup$

There are two ways to look at dot products:

  • The scalar product of two vectors $(v_1,...,v_n)$ and $(w_1,...,w_n)$ can be simply defined as the sum $v_1w_1+\cdots+v_nw_n$, and so its a scalar by definition, or as you learned it
  • as a special kind of matrix multiplication.

In the latter case, the result is indeed a $(1\times 1)$-matrix, and as the comments already stated, its just a convention to call this a scalar, as "the only difference are the brackets". Of course, when you view your result as a matrix, you can only multiply it by $(1\times m)$-matricies. But here is the clue: the space of $(1\times 1)$-matricies is (for all practical purpose) equivalent to the space of scalars. This means, there is a bijective linear map from $K$ to $K^{1\times1}$, even an algebra isomorphism. So, to use the result in more general fashion, we allow us to interpret it as a scalar. Further, you often use the result of a dot product only to multiply it with vectors. And vectors are not so far from the $(1\times m)$-matricies where you are allowed to use your $(1\times1)$-matricies anyway.

$\endgroup$
-1
$\begingroup$

A dot product is not really a scalar, but it behaves just like one. In math we call that an ISOMORPHISM. For every dot product result, there is a corresponding real number that you get by simply removing the brackets. All of the operations you do with the 1x1 matrix correspond to the same operations done with a real number. Multipliction of a matrix by a 1x1 matrix is defined as multiplcation by the corresponding scalar.

Recommend you look up "isomorphism" in a math dictionary. To do this, you may need to brush up on your German language skills.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.