2
$\begingroup$

What does the degrees of freedom parameter $n$ mean intuitively in a Wishart distribution $\mathcal{W}_p(\mathbf{V},n)$? Does it have any relation to the covariance of different dimensions of the resulting covariance matrix? Why is $n==p$ called a non-informative prior?

$\endgroup$
4
$\begingroup$

If we consider the expectations of a covariance matrix $\Sigma^{-1}$ under out prior assumptions that is follows an inverse-Wishart distribution, we see $E(\Sigma^{-1})=nV$ for inverse covariance matrix $V$. Essentially the degrees of freedom parameter arises from statisticians assuming it is a (positive) integer $\Bbb Z^+$, which means we have a multivariate generalisation of the $\chi_\nu^2$ distribution. Note they do not have to be integers.

You will find that as the Chi-squared distribution describes the sums of squares of $n$ draws from a univariate normal distribution, the Wishart distribution represents the sums of squares (and cross-products) of $n$ draws from a multivariate normal distribution.

Hence intuitively for some dimension $p$, if we want to assume:

(1) We are confident about the true covariance matrix being near some covariance matrix $\Sigma_0$, then we can choose a large value for $n$ say $p+10$ and set $V=[n-p-1]\Sigma_0$, so the distribution of our prior $\Sigma^{-1}$ is concentrated around $\Sigma_0$.

(2) We are not too sure about the true covariance matrix being near some covariance matrix $\Sigma_0$, then we can choose a small value for $n$ say $p+2$ and set $V=\Sigma_0$, so the distribution of our prior $\Sigma^{-1}$ is only loosely centered around $\Sigma_0$.

As $n$ decreases to $p$, the distribution becomes increasingly loosely centered around $\Sigma_0$. Hence it will be non-informative when the degrees of freedom is equal to the dimension/number of parameters. This is also clear since the first moment of a Wishart distribution does not exist unless $n>p+1$, with an improper distribution at $n=p$.

$n$ simply then deals with how much we assume about the distribution of the covaraince. But it also happens that if $\Sigma^{-1}$ follows a Wishart distribution, then so does any sub-matrix. In fact I think it can be shown that the sub-matrix degrees of freedom is independent of $p$, the dimension. So technically the degrees of freedom for the sub-matrix parallels that of the original matrix.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.