I am interested in partial derivative of the following term w.r.t $x_1$

$$\mathbf g = \begin{bmatrix}x_1-k_1 ,& x_2-k_2, & x_3-k_3 \end{bmatrix} \begin{bmatrix}s_{11}& s_{12} & s_{13} \\ s_{21}& s_{22} & s_{23} \\s_{31}& s_{32} & s_{33} \end{bmatrix}^{-1} \begin{bmatrix}x_1-k_1 \\ x_2-k_2 \\ x_3-k_3 \end{bmatrix}$$ My real problem is to partially differentiate Gaussian function w.r.t $\mathbf x$ which is defined as $$ f = \exp\left(-\frac{1}{2}({\mathbf x}-{\boldsymbol\mu})^\mathrm{T}{\boldsymbol\Sigma}^{-1}({\mathbf x}-{\boldsymbol\mu})\right)$$

$\Sigma$ is positive semidefinite symmetric matrix

Here I am stuck with the derivative inside $\exp(.)$ term. For $\mathbf{x}\in \mathbb{R}^2$, I could solve it as follows

$$\begin{align} \mathbf g &= \begin{bmatrix}x_1-k_1 ,& x_2-k_2 \end{bmatrix}\begin{bmatrix}\lambda_{11}& \lambda_{12} \\ \lambda_{21}& \lambda_{22}\end{bmatrix}\begin{bmatrix}x_1-k_1 \\ x_2-k_2\end{bmatrix} \\ & = \lambda_{11}(x_1-k_1)^2+(\lambda_{12}+\lambda_{21})(x_1-k_1)( x_2-k_2)+\lambda_{22}(x_2-k_2)^2 \\ \frac{\partial \mathbf g }{\partial x_1} & = 2\lambda_{11}(x_1-k_1)+(\lambda_{12}+\lambda_{21})(x_2-k_2) \end{align} $$

where, $\begin{bmatrix}\lambda_{11}& \lambda_{12} \\ \lambda_{21}& \lambda_{22}\end{bmatrix} = \begin{bmatrix}s_{11}& s_{12}\\ s_{21}& s_{22} \end{bmatrix}^{-1} $

But as the dimension increases it becomes complicated. Can someone guide me how differentiation of matrices can be carried out . I am looking something of this sort and even before that I would like to know, is it possible?.

$$\frac{\partial \mathbf g}{\partial x_1} = \frac{\partial}{\partial x_1} (\mathbf x - \mathbf k)^{\rm T}\Sigma^{-1}\mathbf x + \mathbf x ^{\rm T}\Sigma^{-1}\frac{\partial}{\partial x_1}(\mathbf x - \mathbf k)$$


For convenience, let $$\eqalign{ y &= x-k \cr M &= M^T = \Sigma^{-1}\cr }$$ Write the function in terms of these variables and the Frobenius (:) Inner Product and find its differential $$\eqalign{ g &= M:yy^T \cr\cr dg &= M:(dy\,y^T+y\,dy^T)\cr &= (M+M^T)y:dy \cr &= 2\,My:dy \cr &= 2\,\Sigma^{-1}(x-k):dx \cr }$$ Since $dg=\big(\frac{\partial g}{\partial x}:dx\big),\,$ the gradient is $$\eqalign{ \frac{\partial g}{\partial x} &= 2\,\Sigma^{-1}(x-k) \cr }$$ To find the derivative wrt $x_1$ dot the gradient with the $1^{st}$ basis vector $$\eqalign{ \frac{\partial g}{\partial x_1} &= e_1^T \,\frac{\partial g}{\partial x} \cr &= 2\,e_1^T\,\Sigma^{-1}(x-k) \cr }$$

  • $\begingroup$ thanks @frank, I am evaluating your answer, but you mean $ y = (\mathbf x - \mathbf k)$ (more precisely) $\endgroup$ – pkj May 29 '16 at 19:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.