I am trying to get a feel for the Normal-Inverse-Wishart conjugate prior, which I have started to use, sparingly, in my work, where I am trying to cluster multivariate normal data.

As Wikipedia helpfully points out the NIW is the conjugate prior to the MVN distribution.

What I am trying to get a feel for is how the hyperparameters of the $\mathcal{NIW}(\mu, \lambda, \Psi,\nu)$ affect the cluster properties.


Here is some MVN data with some clusters:

enter image description here

What I am wondering is; if e.g. I were to use a Dirichlet-process mixture model to do some good old clustering, how do my choices of hyperparameters affect my clusters? What makes them highly discriminative (i.e. lots of them with strict boundaries)? What makes them smooth A LOT (i.e. having just say three clusters which between them classify all the data, regardless if this is correct or not)?

$\mu$: I understand that it is common to set this to the sample mean, but what if you don't?

$\lambda$: apparently this is the mean fraction which works as a smoothing parameter (I do not know what this means)

$\Psi$: is the degrees of freedom which is set to the number of dimensions of the data (but what if I set it higher, I cannot go lower, I know that, but higher?

$\nu$: is the pairwise deviation product which is set to the $D \times D$ identity matrix multiplied by a constant. But how does the choice of constant affect my clustering?

Naturally I know that I could have found all of this out empirically, but I have horrendous amounts of data, so would very much appreciate some intuition about the hyperparameter set

$\theta =\{\mu, \lambda, \Psi,\nu\}$.

Equally, running through the maths is obviously possible too, and I am doing that, but as the title says; I just want some advise from more experienced users.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.