Why can I juggle with my products that way?

Context: I am interested in building the matrix that correspond to the operation of projecting a vector $v$ onto a direction $n$. Since $n$ has a norm of 1, the projection of $v$ onto $n$ is $w=(v\cdot n) n$ where "$\cdot$" is the dot product.

Since both the dot and the scalar product commute, I rewrite it $w=n(n\cdot v)$ in order to have $v$ in the end.

The dot product can be replaced with a matrix product: $w=n(n^T \times v)$.

$n^T$ is a row matrix, $v$ a column matrix, and "$\times$" the matrix product. The result of a matrix product is a matrix, in this case 1-by-1, which still somehow works as a scalar.

Then, if I decide that I can associate this way: $w = (n \times n^T) \times v$ then I have found the matrix that I want, it is $M = n \times n^T$.

I know it works because of the brute-force approach I took to test it. If I write down $w_1=n(n.v)$ and $w_2=(n \times n^T) \times v$, I get $w_1=w_2$.

I don't understand why this works. Here, we have three types of products: the dot products between two vectors (column matrices), the matrix product between two matrices, and a scalar product that multiplies a matrix/vector by a scalar. How come I can associate them that way?

• @rschwieb $(n \times n^t)$ is not a scalar here, it's a square matrix, the matrix that I am actually looking for. I'm sorry if my notations are confusing, three sorts of products and not enough symbols. Jun 24, 2013 at 17:16
• @rschwieb : Isn't $w_2$ a matrix times $v,$ and not a scalar times $v?$ Jun 24, 2013 at 17:16
• @Niriel and Will Oh sorry, I didn't catch on you had picked column vectors. I'll review my comments. Thanks for telling me. Jun 24, 2013 at 17:20
• @Niriel In this case, it turns out less symbols would probably be clearer :) Juxtaposition is good for both matrix multiplication and scalar-times-matrix. The difference between scaling and matrix multiplication is usually clear if you use regular letters for vectors and Greek letters for scalars. It would be good to use the dot product symbol, though: use \cdot within latex to get that. If people just look at your symbols before reading the question, I'm sure they'll think you're talking about cross products :) Jun 24, 2013 at 17:36

This is a manifestation of the associativity of matrix multiplication, $$A(BC)=(AB)C.$$ Here you have a $3\times 1$ matrix, $n,$ a $1\times 3$ matrix $n^T,$ and and a $3\times1$ matrix $v.$

The product $nn^Tv$ can be computed with either grouping, just as any matrix product can.

• +1 for getting to the heart of the matter earlier. I just wanted to ramble a bit in my answer... Jun 24, 2013 at 17:32

(Added forewarning: I used row vectors rather than column vectors. I hope that's not an impediment!)

Both the dot product and the scalar product can be written in terms of matrix multiplication. (I prefer to use juxtaposition for matrix multiplication, rather than $\times$, which could be confused with the cross product.)

As you already knew, if $v$ and $w$ are $1\times n$-vectors, then $v\cdot w=v(w^\top)=w\cdot v=w(v^\top)$.

If $M$ is any $m\times n$ matrix and $\lambda$ is any scalar, then $\lambda M=(\lambda I_m)M=M(\lambda I_n)$, where $I_k$ is the $k\times k$ identity matrix.

Since products of matrices are all associative when they are defined, you can move the parentheses around however you like. Will Orrick was the earlier one to point this out, but it doesn't hurt to repeat: If $A, B, C$ are matrices of sizes $m\times n$, $n\times k$ and $k\times j$, then $(AB)C=A(BC)=ABC$.

The ability of scalars to commute with matrices also appears: $(w w^T) v=((w\cdot w) I_1)v=v((w\cdot w) I_n)=v(ww^\top)$. Notice in the last step, the parentheses can't be moved to be $(vw)w^\top$ because that product is not defined (the matrices are the wrong sizes: $v$ and $w$ are both $1\times n$ matrices.) But $(ww^\top)v=w(w^\top v)$ since all of these matrix multiplications are defined.

If you haven't learned already, then you'll probably learn soon that matrices express linear functions from $F^m\to F_n$. Compositions of such functions is always associative, so it makes sense that the matrices that go with them should behave the same way.

• I am still kind of stuck. If I use two different symbols for the scalar product ($\times$) and the matrix product (juxtaposition) to make sure I'm not forgetting anything, I don't get much further. $w = n \times (n^T v)$ becomes $w = n ((n^T v) \times I)$ and I don't know how I can get $n^T$ out of there. Jun 24, 2013 at 17:28
• @Niriel You've got $(n^\top v)\times I$, which I think I'm suppose to interpret as the dot product of $n$ and $v$ with $I$... what's $I$? And $n\times(n^\top v)$ would be a vector dot product with a scalar... typo? Jun 24, 2013 at 17:39
• No more dot product here. $(n^T v)$ is a scalar that multiplies an identity matrix $I$ of the size that matches the number of rows of $n$. Jun 24, 2013 at 17:42
• @Niriel Ah! Yeah I see what you mean now. I'm not sure where we are headed... what you wrote has no problems! Jun 24, 2013 at 17:56
• @Niriel Did you consider that $n\times (n^\top v)=n(n^\top v)$ in this case? The scalar multiplication matches the matrix multiplication on the right of $-\times 1$ matrices matches since then the scalar is a $1 \times 1$ matrix. The $I$ in this case is $I_1$. It shouldn't be $I_n$, because then the matrix dimensions aren't matching. (We're using your column vectors, right?) Jun 24, 2013 at 18:03

Matrix multiplication acts on vectors. You can think of matrix multiplication as a composition of linear maps. Hence, the action follows from $(f\circ g)(v)=f(g(v))$. In fact, the reason the construction of a matrix product is so particular is because it's an algorithm for combining the components from the two matrices so as to produce a matrix that performs their composition as linear maps.