0
$\begingroup$

I'm trying to understand a linear algebra proof involving different kinds of products (from the book "Reinforcement Learning: An Introduction"). The part I'm struggling with is the following identity:

$(R + w^\top x_{t+1} - w^\top x_t) x_t = R x_t - x_t (x_t - x_{t+1})^\top w$

Here $R$ is a real number and $x_t, x_{t+1}, w$ are n-dimensional vectors of real numbers.

The equation mixes different kinds of products (dot, scalar, outer, matrix-by-vector multiplication) but there is no indication which is which. The next line in the proof defines a matrix $A = x_t (x_t - x_{t+1})^\top$ so it appears that they use an identity

$(w^\top \cdot x_{t+1} - w^\top \cdot x_t) x_t = (x_t \otimes \left( x_{t+1} - x_t)^\top \right) w$

Where on the right side we multiply the matrix resulting from the outer product by the vector $w$.

Do I understand this correctly? Why is this identity true?

$\endgroup$
1
  • 2
    $\begingroup$ There is only one operation involved: row by column product (i.e., matrix multiplication). Which is associative. $\endgroup$
    – Giulio R
    Commented Nov 24, 2022 at 13:54

1 Answer 1

Reset to default
1
$\begingroup$

If $\ w,v,u\ $ are $\ n\times1\ $ matrices then $\ w^Tv=v^Tw\ $, and $$ \big(w^Tv\big)\,u=\big(v^Tw\big)\,u $$ is an $\ n\times1\ $ matrix whose $\ i^\text{th}\ $ entry is $\ (v^Tw)\,u_i\ $.

By the associativity of matrix multiplication, $$ \big(uv^T\big)\,w=u\big(v^Tw\big)\ . $$ Both sides of this identity are $\ n\times1\ $ column vectors, and the expression on its right is the product of the $\ n\times1\ $ matrix $\ u\ $ and the $\ 1\times1\ $ matrix $\ v^Tw\ $. The $\ i^\text{th}\ $ entry of this product is therefore also $\ (v^Tw)\,u_i\ $. Thus all these expressions represent the same $\ n\times1\ $ matrix. In particular, $$ \big(w^Tv\big)\,u=\big(uv^T\big)\,w\ . $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .