0
$\begingroup$

If $\mathbf{x}$ is a vector and $Q$ a symmetric Matrix; Does $Q\mathbf{x} = \mathbf{x}^T Q$ ?

EDIT :

I have this: $$\frac{1}{2}\left(Q\mathbf{x} + \mathbf{x}^TQ\right) + \mathbf{c},$$ where $\mathbf{c}$ is also a vector, and I want to obtain $Q\mathbf{x}+\mathbf{c}$.

I thought that was the answer to my question, but it doesn't seem to be. Do you see a way to obtain this?

EDIT : Well Then I believe I made an error prior that, I can explain what I have done before to see if it's the source of the error.

I am trying to find the gradient of the function $$\frac{1}{2}\left(\mathbf{x}^TQ\mathbf{x}\right) + \mathbf{c}\mathbf{x}$$

that's how I got the first answer by finding the gradient.

$\endgroup$
2
  • $\begingroup$ Consider the case where $Q$ is the identity matrix. $\endgroup$ Jan 18, 2011 at 5:48
  • $\begingroup$ After the edit: This is still nonsense. If $Q$ is $n\times n$, then for $Q\mathbf{x}$ and $\mathbf{x}^TQ$ to both make sense with x a vector you need $\mathbf{x}$ to be $n\times 1$. Then $Q\mathbf{x}$ is $n\times 1$, and $x^T\mathbf{Q}$ is $1\times n$; the only way you can add them, and add them to a $k\times 1$ vector $\mathbf{c}$, is for $n=k=1$ and for you to be working with numbers, not matrices and vectors; in which case you already have $Q\mathbf{x}+\mathbf{c}$, because $\frac{1}{2}(Q\mathbf{x}+\mathbf{x}^TQ)+\mathbf{c} = \frac{1}{2}(Qx+Qx)+c = Qx+c$. $\endgroup$ Jan 18, 2011 at 21:41

3 Answers 3

4
$\begingroup$

What you have is $f(x) = \frac{1}{2}x^TQx + cx$ and you want to evaluate $\frac{df(x)}{dx}$

First make sure to get the dimensions of all the vectors and matrices in place.

Here $x \in \mathbb{R}^{n \times 1}$, $Q \in \mathbb{R}^{n \times n}$ and $c \in \mathbb{R}^{1 \times n}$, $f(x) \in \mathbb{R}$

This will help you to make sure that your arithmetic operations are performed on vectors of appropriate dimensions.

Now lets move on to the differentiation.

All you need to know are the following rules for vector differentiation.

$$\frac{d(x^Ta)}{dx} = \frac{d(a^Tx)}{dx} = a$$ where $x,a \in \mathbb{R}^{n \times 1}$.

Note that $x^Ta = a^Tx$ since it is a scalar.

(Some people follow a different convention i.e. treating the derivative as a row vector instead of a column vector. Make sure to stick to your convention and you will end up with the same conclusion in the end)

Make use of the above two results to get,

$$\frac{d(x^TQx)}{dx} = (Q^T + Q)x \text{ (use chain rule here)}$$

$$\frac{d(cx)}{dx} = c^T$$

So, $\frac{df(x)}{dx} = \frac{1}{2}((Q^T + Q)x) + c^T$ and since $Q$ is symmetric, we get $Q^T = Q$ and hence we get $$\frac{df(x)}{dx} = Qx + c^T$$

$\endgroup$
2
  • $\begingroup$ I'm trying to figure out how you ended up with (QT + Q)x with the chain rule. $\endgroup$
    – DogDog
    Jan 19, 2011 at 0:25
  • $\begingroup$ @Apoc: First treat $x^TQ$ as a constant vector, so we will get $Q^Tx$, now treat $Qx$ as a constant vector and we will get $Qx$. Hence, $\frac{d(x^TQx)}{dx} = Q^Tx + Qx$ $\endgroup$
    – user17762
    Jan 19, 2011 at 0:29
3
$\begingroup$

No, the dimensions don't match. Qx is a column vector and $x^TQ$ is a row vector. But you are close-what can you do to fix this?

$\endgroup$
1
$\begingroup$

We know that $Qx = Q^{T}x$. That is all we know. But $Q^{T}x$ is not necessarily equal to $x^{T}Q$ since the dimensions don't match.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .