Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Given a covariance matrix $$\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)$$ where $\sigma,\lambda \in \mathbb{R}$ (both positive), $I$ is the $n \times n$ identity matrix and $K_{\theta}^{-1}$ is known. Moreover, $K_{\theta}^{-1}$ is positive definite and symmetric. I want to find an expression for $\lambda$ that satisfies

$$\frac{\partial}{\partial \lambda} (2\pi)^{n/2}\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-\frac{1}{2}}\exp\left[-\frac{1}{2}y^T\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y\right] = 0$$

Here, $n$ is fixed and $y$ is given.

I've been spending a lot of time trying to find an optimal $\lambda$. At the moment I'm not even sure it can be done but I would love to be shown wrong. Here's what I did so far. Since we can also find the derivative of the log of the expression, we have

$$\frac{\partial}{\partial \lambda}\left[ -\frac{1}{2}\log\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right) - \frac{1}{2}y^T\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y\right] = 0$$ or $$\frac{\partial}{\partial \lambda}\left[\log\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)\right] + \frac{\partial}{\partial \lambda}\left[y^T\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y\right]=0$$

Concerning the first term, I've found that for a symmetric matrix $M = M(\alpha)$ for some scalar $\alpha$, it holds that $$\frac{\partial}{\partial \alpha} \log\det M(\alpha) = \text{tr}\left(M(\alpha)^{-1}\frac{\partial}{\partial \alpha}M(\alpha)\right)$$ Applying this to $\frac{\partial}{\partial \lambda}\left[\log\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)\right]$ gives $$\frac{\partial}{\partial \lambda}\left[\log\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}\right] = \frac{-1}{\lambda^2}\text{tr}\left[\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}K_{\theta}^{-1}\right]$$ which I can simplify to $$\frac{-1}{\lambda}\text{tr}\left[\left(\lambda\sigma K_{\theta} + I \right)^{-1}\right]$$

Concerning the derivation of the second term, I've asked about this in a previous question, which you can read here: Finding the scalar derivative of a matrix product. It turns out that we can simplify $$\frac{\partial}{\partial \lambda}\left[y^T\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y\right]$$ to $$y^T\left(2\lambda\sigma^2 I + K_{\theta}^{-1} + \lambda^2 \sigma^4 K_{\theta}\right)^{-1}y$$ Thus, I have $$\frac{-1}{\lambda}\text{tr}\left[\left(\lambda\sigma^2 K_{\theta} + I \right)^{-1}\right] + y^T\left(2\lambda\sigma^2 I + K_{\theta}^{-1} + \lambda^2 \sigma^4 K_{\theta}\right)^{-1}y = 0$$ which I would like to solve for $\lambda$. Is there any way to do this analytically or am I doomed to numerical methods? Thanks in advance.

share|cite|improve this question

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.