I would like some hints on how to derive the following function with respect to the matrix $X$:

$$ f(X)=\left\| y - \sum_k (h_k X^k)x \right\|_2^2$$

where $X$ is a matrix, $x$ and $y$ are two given vectors and $h_k$ is a (given) scalar coefficient that weights the matrix. This is the composition of a convex function (the norm) with a non affine function. I want to linearize the function with respect to X, so I would like to compute the derivative. Some hints?

Thank you.

  • 1
    $\begingroup$ What you have written is a function of $y, x$, and $\{h_k\}$. But it is not a function of $X$, since $X$ is a dummy variable in the expression. Did you perhaps mean $\min_x$ instead? $\endgroup$ – Paul Sinclair Jan 28 at 4:10
  • $\begingroup$ @PaulSinclair, I edited the question by explaining what each variable represents. All is given except the matrix $X$, which is the optimization variable. So the function is $f(X)$ $\endgroup$ – Alberto Jan 28 at 10:08
  • $\begingroup$ If you are talking about $f(X)$, why did you put an extraneous "$\min_X$ in front? It serves no purpose except to seriously obfuscate. $\endgroup$ – Paul Sinclair Jan 28 at 14:44
  • $\begingroup$ @PaulSinclair you are right. $\endgroup$ – Alberto Jan 28 at 15:28

If you are just trying to linearize $f(X)$ around $X= 0$, this is easily accomplished

$$f(X) =\left(y - \sum_k h_k X^kx\right)^T\left(y - \sum_k h_k X^kx\right)\\ =\left(y^T - \sum_k h_kx^T(X^T)^k\right)^T\left(y - \sum_k h_k X^kx\right)\\ =y^Ty - \sum_k h_ky^TX^kx - \sum_k h_kx^T(X^T)^ky + \sum_j\sum_k h_jh_k x^T(X^T)^jX^kx\\ = y^Ty + h_0^2x^Tx - h_0(y^Tx + x^Ty) - h_1(y^TXx + x^TX^Ty) + 2h_0h_1x^TXx + O(X^2)$$

Linearization just means ignoring higher order terms, so the linearization for $X$ near $0$ is $$y^Ty + h_0^2x^Tx - h_0(y^Tx + x^Ty) - h_1(y^TXx + x^TX^Ty) + 2h_0h_1x^TXx$$ If there are no terms for $k = 0$, it simplifies to $$y^Ty - h_1(y^TXx + x^TX^Ty)$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.