# Derivative of a composite function with respect to a matrix

I would like some hints on how to derive the following function with respect to the matrix $$X$$:

$$f(X)=\left\| y - \sum_k (h_k X^k)x \right\|_2^2$$

where $$X$$ is a matrix, $$x$$ and $$y$$ are two given vectors and $$h_k$$ is a (given) scalar coefficient that weights the matrix. This is the composition of a convex function (the norm) with a non affine function. I want to linearize the function with respect to X, so I would like to compute the derivative. Some hints?

Thank you.

• What you have written is a function of $y, x$, and $\{h_k\}$. But it is not a function of $X$, since $X$ is a dummy variable in the expression. Did you perhaps mean $\min_x$ instead? – Paul Sinclair Jan 28 at 4:10
• @PaulSinclair, I edited the question by explaining what each variable represents. All is given except the matrix $X$, which is the optimization variable. So the function is $f(X)$ – Alberto Jan 28 at 10:08
• If you are talking about $f(X)$, why did you put an extraneous "$\min_X$ in front? It serves no purpose except to seriously obfuscate. – Paul Sinclair Jan 28 at 14:44
• @PaulSinclair you are right. – Alberto Jan 28 at 15:28

If you are just trying to linearize $$f(X)$$ around $$X= 0$$, this is easily accomplished
$$f(X) =\left(y - \sum_k h_k X^kx\right)^T\left(y - \sum_k h_k X^kx\right)\\ =\left(y^T - \sum_k h_kx^T(X^T)^k\right)^T\left(y - \sum_k h_k X^kx\right)\\ =y^Ty - \sum_k h_ky^TX^kx - \sum_k h_kx^T(X^T)^ky + \sum_j\sum_k h_jh_k x^T(X^T)^jX^kx\\ = y^Ty + h_0^2x^Tx - h_0(y^Tx + x^Ty) - h_1(y^TXx + x^TX^Ty) + 2h_0h_1x^TXx + O(X^2)$$
Linearization just means ignoring higher order terms, so the linearization for $$X$$ near $$0$$ is $$y^Ty + h_0^2x^Tx - h_0(y^Tx + x^Ty) - h_1(y^TXx + x^TX^Ty) + 2h_0h_1x^TXx$$ If there are no terms for $$k = 0$$, it simplifies to $$y^Ty - h_1(y^TXx + x^TX^Ty)$$