Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I have a normal distribution, the posterior for the variance is the inverse Chi-square distribution assuming the same is used as a conjugate prior. But what if my data has extra noise added so that the observed sample variance is the sum of the population variance and my extra noise variance? But then the poisterior for the variance is different. Is there a name for that distribution?

You can't just subtract the noise term because you can end up with negative values. It is similar to the Skellam distribution of the difference of two Poisson variables in this way.

I am really interested in this from a Gibbs sampler point of view. I would like to draw the variance from the conditional posterior if possible. If that isn't easy I can fall back on Metropolis Hastings, I suppose.

share|cite|improve this question
If the population variance you want to estimate is $\sigma^2$ and the error variance is $\tau^2$, then the variance of the actual observation is $\sigma^2+tau^2$ (assuming uncorrelatedness of the two). But if the population variance you want to estimate is $\tau^2$ and the error variance is $\sigma^2$, then the variance of the actual observation is still $\sigma^2+tau^2$. If you observe only their sum, you can't tell the difference. On the other hand, if you observe several realizations of the sum in which the errors are independent but the observation from the population you're... – Michael Hardy May 12 '12 at 0:12
...trying to understand are the same, then you can do something. Say you've get $X_i + \varepsilon_{i,j}$ for $i=1,\ldots,n$ and $j=1,\ldots,m_i$. Then you have a variance-components problem. Estimates by the method of moments can actually give you negative values for the variances (at least in similar problems; I'm unsure about this particular one). MLEs of course cannot. Nor can Bayesian estimates. – Michael Hardy May 12 '12 at 0:15
But anyway, could you clarify? – Michael Hardy May 12 '12 at 0:15

So yes, that is indeed the Maximum Likelihood Estimator. But I want the full distribution function. To make this more concrete... I am trying to do a Gaussian mixture model in 2D. If there is additive random background noise, you can model that fine. But if the positions are measured with some error as they always are, the mixture model solution will just give you the probability distribution of the smoothed data. But you can't just subtract off the spatial uncertainty variance. That might give you the right ML estimated as long as they don't go negative (or not) but will not be the right distribution function. The right distribution function is the deconvolution of the likelihood with the distribution of the spatial noise. Maybe this is ill-defined as most deconvolution problems are but I was hoping that there is a solution given that we are using a model which should act to regularize the inversion. I am sure there will be a prior that needs to be specified especially to constrain the highest frequency part which will be convolved away.

share|cite|improve this answer
It looks like there are sufficient statistics for this and it is an exponential family distribution. So you can put a conjugate prior on it and get a simple posterior. However that prior doesn't rule out negative values and so this conjugate prior is not as useful as they usually are. If you use another reasonable prior, you can get a sensible answer. However, it doesn't come out as a common distribution that can be sampled from simply. Of course, you can always use a rejection method to sample so it isn't so bad. – David Johnston May 12 '12 at 23:48

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.