1
$\begingroup$

In my current project, I have a function $g(\mathbf{x})$ which I scale with a multivariate exponential term. Now I require a specific partial derivative of this function, but the involvement of matrix operations in the exponential term crosses the threshold into matrix calculus, somewhat beyond my limited mathematics background. I have already made an attempt at this partial derivative, but am not entirely confident that my derivation is correct. Perhaps a second pair of eyes might help - could you tell me whether this derivation is correct, and if not, where I have made a mistake?


Assume I have the following function:

$$g(\mathbf{x})=f(\mathbf{x})\exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) \tag{1}$$

where $\mathbf{x}=[x_1,x_2,...,x_D]^T\in\mathbb{R}^D$, $f(\mathbf{x}):\mathbb{R}^D\rightarrow\mathbb{R}$, and $\Sigma$ is a covariance matrix. I wish to take the partial derivative of this expression with regards to a map component $x_i, 1 \leq i \leq D$, i.e.:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})=\frac{\partial}{\partial x_i}\left(f(\mathbf{x})\exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x})\right) \tag{2}$$

Applying the chain rule yields:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})=\frac{\partial f(\mathbf{x})}{\partial x_i} \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) + f(\mathbf{x}) \frac{\partial \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x})}{\partial x_i} \tag{3}$$

The first summand is complete, but we can expand the second summand further. Using the derivative rule for exponential functions $exp(f(x))'=exp(f(x))f'(x)$ we obtain:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})=\frac{\partial f(\mathbf{x})}{\partial x_i} \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) + f(\mathbf{x}) \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) \frac{\partial -\mathbf{x}^T\Sigma^{-1}\mathbf{x}}{\partial x_i} \tag{4}$$

Here I am not entirely sure... treating $\frac{\partial}{\partial x_i}$ as a left-side matrix operator, it only affects the left-most $\mathbf{x}^T$. If we have:

$$\frac{\partial \mathbf{x}^T}{\partial x_i}=[0,...,1_i,...0]^T$$

where $1_i$ denotes a $1$ in the $i$-th component of the vector, then we have:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})=\frac{\partial f(\mathbf{x})}{\partial x_i} \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) - f(\mathbf{x}) \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) [0,...,1_i,...0]^T\Sigma^{-1}\mathbf{x} \tag{5}$$

which can be simplified to:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})= \exp(\mathbf{x}^T\Sigma^{-1}\mathbf{x}) \left( \frac{\partial f(\mathbf{x})}{\partial x_i} - f(\mathbf{x}) [0,...,1_i,...0]^T\Sigma^{-1}\mathbf{x} \right) \tag{6}$$

Is this derivation correct?

$\endgroup$

1 Answer 1

1
$\begingroup$

$ \def\p{{\partial}} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\i#1{#1^{-1}} $For typing convenience define the matrix variable $$M=\Sigma^{-1}$$ and use a colon as a convenient product notation for the trace, i.e. $\;A:B = {\rm Tr}(AB^T)$

Then take the logarithms of the functions before differentiating $$\eqalign{ \log g &= \log f - M:xx^T \\ \i{g}dg &= \i{f}df - M:\left(dx\,x^T+x\,dx^T\right) \\ dg &= \frac gf\left(\grad{f}{x}\right):dx - g\left(M+M^T\right)x:dx \\ \grad{g}{x} &= \frac gf\left(\grad{f}{x}\right) - 2gMx \\ }$$ This agrees with your result, except that you are missing a factor of two on the rightmost term.
So you made a mistake differentiating the $\left(x^T\Sigma^{-1}x\right)$ quadratic form.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .