# Scalar and dot product associativity

I'm trying to understand a linear algebra proof involving different kinds of products (from the book "Reinforcement Learning: An Introduction"). The part I'm struggling with is the following identity:

$$(R + w^\top x_{t+1} - w^\top x_t) x_t = R x_t - x_t (x_t - x_{t+1})^\top w$$

Here $$R$$ is a real number and $$x_t, x_{t+1}, w$$ are n-dimensional vectors of real numbers.

The equation mixes different kinds of products (dot, scalar, outer, matrix-by-vector multiplication) but there is no indication which is which. The next line in the proof defines a matrix $$A = x_t (x_t - x_{t+1})^\top$$ so it appears that they use an identity

$$(w^\top \cdot x_{t+1} - w^\top \cdot x_t) x_t = (x_t \otimes \left( x_{t+1} - x_t)^\top \right) w$$

Where on the right side we multiply the matrix resulting from the outer product by the vector $$w$$.

Do I understand this correctly? Why is this identity true?

• There is only one operation involved: row by column product (i.e., matrix multiplication). Which is associative. Nov 24, 2022 at 13:54

If $$\ w,v,u\$$ are $$\ n\times1\$$ matrices then $$\ w^Tv=v^Tw\$$, and $$\big(w^Tv\big)\,u=\big(v^Tw\big)\,u$$ is an $$\ n\times1\$$ matrix whose $$\ i^\text{th}\$$ entry is $$\ (v^Tw)\,u_i\$$.
By the associativity of matrix multiplication, $$\big(uv^T\big)\,w=u\big(v^Tw\big)\ .$$ Both sides of this identity are $$\ n\times1\$$ column vectors, and the expression on its right is the product of the $$\ n\times1\$$ matrix $$\ u\$$ and the $$\ 1\times1\$$ matrix $$\ v^Tw\$$. The $$\ i^\text{th}\$$ entry of this product is therefore also $$\ (v^Tw)\,u_i\$$. Thus all these expressions represent the same $$\ n\times1\$$ matrix. In particular, $$\big(w^Tv\big)\,u=\big(uv^T\big)\,w\ .$$