Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm trying to minimize the (negative) multivariate normal log likelihood (dropping constants):

$$ \log |\boldsymbol\Sigma|\,+(\mathbf{x}-\boldsymbol\mu)^{\rm T}\boldsymbol\Sigma^{-1}(\mathbf{x}-\boldsymbol\mu)$$


$$ \Sigma_{ij} = \sigma_1 \cdot \mathcal{I}\{i = j \} + \sigma_2 \cdot \mathcal{I}\{ L_i = L_j \} + \sigma_3 \cdot f(||L_i - L_j||)$$ as a function of $\{ \sigma_1, \sigma_2, \sigma_3, {\boldsymbol \mu} \}$.

The $L_i$ are known values - in the context of the problem it is the "location" of unit $i$. The function $f(\cdot)$ is a known monotonically decreasing function. $\mathcal{I}(\cdot)$ is an indicator function and $|| \cdot ||$ is a distance measure (say, euclidean distance). Each of the $\sigma_i$ parameters are positive so that $\Sigma$ is certainly positive definite.

In this problem the dimension of $\Sigma$ is very large, say $1000 \times 1000$, so naively evaluating the log-likelihood is relatively computationally expensive ($\approx$ 2 seconds per evaluation). I'm thinking that, since $\Sigma$ has this blocked structure, there may be some way exploit this fact to significantly speed up computation, but I'm having some trouble figuring how/if this will work.

Any tips are appreciated.

Update: I can see that $\Sigma$ can be written as

$$ \Sigma = \sigma_1 {\bf I}+ {\bf C} \otimes {\bf 1} $$

where ${\bf 1}$ is a matrix of $1$s, ${\bf I}$ is the identity, $\otimes$ denotes the Kronecker product, and ${\bf C}$ is an $N \times N$ matrix, where $N$ is the number of "locations" and has the structure

$$ {\bf C}_{nm} = \sigma_2 \cdot \mathcal{I} \{ n = m \} + \sigma_3 \cdot f( \delta_{nm} )$$

where $\delta_{nm}$ denotes the distance between location $n$ and $m$. Still not quite sure this helps me a ton. Will be back with more updates perhaps.

share|cite|improve this question

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.