Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Given a covariance matrix $$\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)$$ where $\sigma,\lambda \in \mathbb{R}$ (both positive), $I$ is the $n \times n$ identity matrix and $K_{\theta}^{-1}$ is known. Moreover, $K_{\theta}^{-1}$ is positive definite and symmetric. I want to find an expression for $\lambda$ that satisfies

$$\frac{\partial}{\partial \lambda} (2\pi)^{n/2}\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-\frac{1}{2}}\exp\left[-\frac{1}{2}y^T\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y\right] = 0$$

Here, $n$ is fixed and $y$ is given.

I've been spending a lot of time trying to find an optimal $\lambda$. At the moment I'm not even sure it can be done but I would love to be shown wrong. Here's what I did so far. Since we can also find the derivative of the log of the expression, we have

$$\frac{\partial}{\partial \lambda}\left[ -\frac{1}{2}\log\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right) - \frac{1}{2}y^T\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y\right] = 0$$ or $$\frac{\partial}{\partial \lambda}\left[\log\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)\right] + \frac{\partial}{\partial \lambda}\left[y^T\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y\right]=0$$

Concerning the first term, I've found that for a symmetric matrix $M = M(\alpha)$ for some scalar $\alpha$, it holds that $$\frac{\partial}{\partial \alpha} \log\det M(\alpha) = \text{tr}\left(M(\alpha)^{-1}\frac{\partial}{\partial \alpha}M(\alpha)\right)$$ Applying this to $\frac{\partial}{\partial \lambda}\left[\log\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)\right]$ gives $$\frac{\partial}{\partial \lambda}\left[\log\det\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}\right] = \frac{-1}{\lambda^2}\text{tr}\left[\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}K_{\theta}^{-1}\right]$$ which I can simplify to $$\frac{-1}{\lambda}\text{tr}\left[\left(\lambda\sigma K_{\theta} + I \right)^{-1}\right]$$

Concerning the derivation of the second term, I've asked about this in a previous question, which you can read here: Finding the scalar derivative of a matrix product. It turns out that we can simplify $$\frac{\partial}{\partial \lambda}\left[y^T\left(\sigma^2 I + \lambda^{-1}K_{\theta}^{-1}\right)^{-1}y\right]$$ to $$y^T\left(2\lambda\sigma^2 I + K_{\theta}^{-1} + \lambda^2 \sigma^4 K_{\theta}\right)^{-1}y$$ Thus, I have $$\frac{-1}{\lambda}\text{tr}\left[\left(\lambda\sigma^2 K_{\theta} + I \right)^{-1}\right] + y^T\left(2\lambda\sigma^2 I + K_{\theta}^{-1} + \lambda^2 \sigma^4 K_{\theta}\right)^{-1}y = 0$$ which I would like to solve for $\lambda$. Is there any way to do this analytically or am I doomed to numerical methods? Thanks in advance.

share|improve this question
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.