1
$\begingroup$

I have found some somewhat similar questions but nothing seems to be quite the same as my problem, but please correct me if I'm wrong.

I have a scalar cost function: \begin{equation} C(\boldsymbol{x}, \boldsymbol{v}) = \frac{1}{2} ||f(\boldsymbol{x}, \boldsymbol{v})||^2_2 \end{equation}

The function $f$ is a vector valued function that operates on a vector $f: \mathbb{R}^p \rightarrow \mathbb{R}^m$

It can be written in terms of a matix-valued function multiplied by a vector: \begin{equation} f(\boldsymbol{x}, \boldsymbol{v}) = G(\boldsymbol{x}) \cdot \boldsymbol{v} \end{equation}

I can use the chain rule to get the equation $\frac{\partial C}{\partial \boldsymbol{x}} = \frac{\partial f}{\partial \boldsymbol{x}} \cdot G(\boldsymbol{x})\cdot \boldsymbol{v}$

In my context the value of $\frac{\partial C}{\partial \boldsymbol{x}}$ and $\boldsymbol{x}$ have been empirically acquired and my goal is to learn the best fit of $\boldsymbol{v}$ from this data (ideally from regression). Where I'm running into confusion is correctly representing $\frac{\partial f}{\partial \boldsymbol{x}}$ and then determining if such a fit is mathematically feasible.

First I'm looking to take the derivative of $f$ with respect to $\boldsymbol{x}$, in terms of $G$ and $\boldsymbol{v}$.

So far this has been my approach: \begin{align} f(\boldsymbol{x}, \boldsymbol{v}) &= G(\boldsymbol{x}) \cdot \boldsymbol{v}\\ &= \begin{bmatrix} v_1 G_{11}+ \ldots + v_n G_{1n} \\ \vdots \\ v_1 G_{m1}+ \ldots + v_n G_{mn} \end{bmatrix}\\ \frac{\partial f}{\partial \boldsymbol{x}}&= \begin{bmatrix} v_1 \frac{\partial G_{11}}{\partial x_1}+ \ldots + v_n \frac{\partial G_{1n}}{\partial x_1} & \cdots & v_1 \frac{\partial G_{11}}{\partial x_p}+ \ldots + v_n \frac{\partial G_{1n}}{\partial x_p}\\ \vdots \\ v_1 \frac{\partial G_{m1}}{\partial x_1}+ \ldots + v_n \frac{\partial G_{mn}}{\partial x_1} & \cdots & v_1 \frac{\partial G_{m1}}{\partial x_p}+ \ldots + v_n \frac{\partial G_{mn}}{\partial x_p} \end{bmatrix}\\ &= \begin{bmatrix} \frac{\partial G}{\partial x_1} & \cdots & \frac{\partial G}{\partial x_p} \end{bmatrix} \begin{bmatrix} \boldsymbol{v} & 0 & \cdots & 0 \\ 0 & \boldsymbol{v} & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \boldsymbol{v} \end{bmatrix}_{np \times p} \end{align}

Is there a more compact way to represent this or some further manipulation I could be exploiting here? And furthermore, can the derivative $\frac{\partial C}{\partial \boldsymbol{x}}$ therefore be expressed as an expression which is linear in either $\boldsymbol{v}$ or some transform of $\boldsymbol{v}$?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.