# Sanity check: Partial derivative of a function with a mutltivariate Gaussian exponential term?

In my current project, I have a function $$g(\mathbf{x})$$ which I scale with a multivariate exponential term. Now I require a specific partial derivative of this function, but the involvement of matrix operations in the exponential term crosses the threshold into matrix calculus, somewhat beyond my limited mathematics background. I have already made an attempt at this partial derivative, but am not entirely confident that my derivation is correct. Perhaps a second pair of eyes might help - could you tell me whether this derivation is correct, and if not, where I have made a mistake?

Assume I have the following function:

$$g(\mathbf{x})=f(\mathbf{x})\exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) \tag{1}$$

where $$\mathbf{x}=[x_1,x_2,...,x_D]^T\in\mathbb{R}^D$$, $$f(\mathbf{x}):\mathbb{R}^D\rightarrow\mathbb{R}$$, and $$\Sigma$$ is a covariance matrix. I wish to take the partial derivative of this expression with regards to a map component $$x_i, 1 \leq i \leq D$$, i.e.:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})=\frac{\partial}{\partial x_i}\left(f(\mathbf{x})\exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x})\right) \tag{2}$$

Applying the chain rule yields:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})=\frac{\partial f(\mathbf{x})}{\partial x_i} \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) + f(\mathbf{x}) \frac{\partial \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x})}{\partial x_i} \tag{3}$$

The first summand is complete, but we can expand the second summand further. Using the derivative rule for exponential functions $$exp(f(x))'=exp(f(x))f'(x)$$ we obtain:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})=\frac{\partial f(\mathbf{x})}{\partial x_i} \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) + f(\mathbf{x}) \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) \frac{\partial -\mathbf{x}^T\Sigma^{-1}\mathbf{x}}{\partial x_i} \tag{4}$$

Here I am not entirely sure... treating $$\frac{\partial}{\partial x_i}$$ as a left-side matrix operator, it only affects the left-most $$\mathbf{x}^T$$. If we have:

$$\frac{\partial \mathbf{x}^T}{\partial x_i}=[0,...,1_i,...0]^T$$

where $$1_i$$ denotes a $$1$$ in the $$i$$-th component of the vector, then we have:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})=\frac{\partial f(\mathbf{x})}{\partial x_i} \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) - f(\mathbf{x}) \exp(-\mathbf{x}^T\Sigma^{-1}\mathbf{x}) [0,...,1_i,...0]^T\Sigma^{-1}\mathbf{x} \tag{5}$$

which can be simplified to:

$$\frac{\partial}{\partial x_i}g(\mathbf{x})= \exp(\mathbf{x}^T\Sigma^{-1}\mathbf{x}) \left( \frac{\partial f(\mathbf{x})}{\partial x_i} - f(\mathbf{x}) [0,...,1_i,...0]^T\Sigma^{-1}\mathbf{x} \right) \tag{6}$$

Is this derivation correct?

$$\def\p{{\partial}} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\i#1{#1^{-1}}$$For typing convenience define the matrix variable $$M=\Sigma^{-1}$$ and use a colon as a convenient product notation for the trace, i.e. $$\;A:B = {\rm Tr}(AB^T)$$
Then take the logarithms of the functions before differentiating \eqalign{ \log g &= \log f - M:xx^T \\ \i{g}dg &= \i{f}df - M:\left(dx\,x^T+x\,dx^T\right) \\ dg &= \frac gf\left(\grad{f}{x}\right):dx - g\left(M+M^T\right)x:dx \\ \grad{g}{x} &= \frac gf\left(\grad{f}{x}\right) - 2gMx \\ } This agrees with your result, except that you are missing a factor of two on the rightmost term.
So you made a mistake differentiating the $$\left(x^T\Sigma^{-1}x\right)$$ quadratic form.