The derivation of gradient of the marginal likelihood is given in http://www.gaussianprocess.org/gpml/chapters/RW5.pdf

But the gradient for the most commonly used covariance function, squared exponential covariance, is not explicitly given.

I am implementing the Rprop algorithm in http://ml.informatik.uni-freiburg.de/_media/publications/blumesann2013.pdf for learning hyper-parameters sigma (signal variance) and h (length). Alas, my implementation is not working well. I have derived the gradients but I am not sure if they are correct.

Can someone point me to a good tutorial / article that explicitly give the expressions for the hyper parameter gradients?


We are looking to maximise the log probability of $lnP(y|x, \theta)$:

$$\ln P(y|x, \theta) = -\frac{1}{2}\ln|K| - \frac{1}{2}y^tK^{-1}y - \frac{N}{2}\ln{2\pi}$$

The three components can be seen as balancing the complexity of the GP (to avoid overfit) and the data fit, with a constant on the end. So the gradient is

$$\frac{\partial}{\partial\theta_i} \log P(y|x, \theta) = \frac{1}{2}y^TK^{-1}\frac{\partial K}{\partial\theta_i}K^{-1}y^T -\frac{1}{2}\mathrm{tr}\left(K^{-1}\frac{\partial K}{\partial\theta_i}\right)$$

So all we need to know is $\frac{\partial K}{\partial\theta_i}$ to be able to solve it. I think you got this far but I wasn't sure so I thought I would recap.

For the case of the RBF/expodentiated quadratic (never call it squared exponential as this is actually incorrect) kernel, under the following formulation:

$$K(x,x') = \sigma^2\exp\left(\frac{-(x-x')^T(x-x')}{2l^2}\right)$$

The derivatives with respect to the hyperparameters are as follows:

$$\frac{\partial K}{\partial\sigma} = 2\sigma\exp\left(\frac{-(x-x')^T(x-x')}{2l^2}\right)$$

$$\frac{\partial K}{\partial l} = \sigma^2\exp\left(\frac{-(x-x')^T(x-x')}{2l^2}\right) \frac{(x-x')^T(x-x')}{l^3}$$

However, often GP libraries use the notation:

$$K(x,x') = \sigma\exp\left(\frac{-(x-x')^T(x-x')}{l}\right)$$

where $\sigma$ and $l$ is confined to only to positive real numbers. Let $l=exp(\theta_1)$ and $\sigma=exp(2\theta_2)$, then by passing in $a,b$ we know are values will conform to this rule. In this case the derivatives are:

$$\frac{\partial K}{\partial\theta_1} = \sigma\exp\left(\frac{-(x-x')^T(x-x')}{2l^2}\right)\left(\frac{(x-x')^T(x-x')}{l^2}\right)$$

$$\frac{\partial K}{\partial \theta_2} = 2 \sigma\exp\left(\frac{-(x-x')^T(x-x')}{2l^2}\right)$$

There is is interesting work carried out by the likes of Mike Osborne looking at marginalising out hyper parameters. However as far as I am aware I think it is only appropriate for low numbers of parameters and isn't incorporated in standard libraries yet. Worth a look all the same.

Another not is that the optimisation space is multimodal the majority of the time so if you are using convex optimisation be sure to use a fare few initialisations.

  • $\begingroup$ Many thanks! This is very helpful. $\endgroup$ – aaronqli Dec 20 '14 at 19:52
  • 2
    $\begingroup$ jpro is right, your answer for $\frac{dK}{dl}$ is incorrect $\endgroup$ – George Aug 25 '16 at 19:27
  • $\begingroup$ Ah sorry about that - I'll fix it when I get home not to confuse people in the future $\endgroup$ – j__ Aug 26 '16 at 13:21

I believe that the derivative of $\frac{\partial K}{\partial l}$, as it was given by j_f is not correct. I think that the correct one is the following (i present the derivation step by step):

$K(x,x') = \sigma^2\exp\big(\frac{-(x-x')^T(x-x')}{2l^2}\big)$
I now call $g(l)=\big(\frac{-(x-x')^T(x-x')}{2l^2}\big)$. So $K=\sigma^2exp\big( g(l) \big)$.

$\frac{\partial K}{\partial l} = \frac{\partial K}{\partial g} \frac{\partial g}{\partial l} = \sigma^2\exp\big(\frac{-(x-x')^T(x-x')}{2l^2}\big) \frac{\partial g}{\partial l}$.

With simple calculations, I finally get:

$\frac{\partial K}{\partial l} = \sigma^2\exp\big(\frac{-(x-x')^T(x-x')}{2l^2}\big) \frac{(x-x')^T(x-x')}{l^3}$.


Maybe there is also an error in the gradient formulation, because in Rasmussen&Williams - Gaussian Process for Machine Learning, p.114, eq 5.9, it is expressed as:

Partial deivatives log marginal likelihood w.r.t. hyperparameters

where the 2 terms have different signs and the y targets vector is transposed just the first time.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.