13
$\begingroup$

In lots of Bayesian papers, people use variational approximation. In lots of them they call it "mean-field variational approximation". Does anyone know what is the meaning of mean-field in this context?

$\endgroup$

3 Answers 3

10
$\begingroup$

I found some intuitions that might answer this; based on the definition of "mean-field" at Wikipedia, mean field theory (MFT also known as self-consistent field theory) studies the behaviour of large and complex stochastic models by studying a simpler model. Such models consider a large large number of small interacting individuals who interact with each other. The effect of all the other individuals on any given individual is approximated by a single averaged effect, thus reducing a many-body problem to a one-body problem.

So basically approximating the inference and learning problem, using independence assumptions and decomposition into several products, brings the notion of "mean-field" approximation.

$\endgroup$
6
$\begingroup$

I believe the mean-field approximation used in mean-field variational Bayes is the assumption that the posterior approximation factorizes over the parameters

$$q(\mathbf{\theta}) = q_1(\theta_1) q_2(\theta_2) \dots q_n(\theta_n)$$

$\endgroup$
3
  • 1
    $\begingroup$ Is it obvious why this factorization is called "mean-field"? $\endgroup$
    – Daniel
    Apr 14, 2017 at 15:08
  • 1
    $\begingroup$ It gets this name from it's origin in Physics. In a system with many components that interact, the many interactions are approximated with a single average interaction - i.e. the mean field. The analogy to Bayesian Variational Inference is that a complicated posterior can be simplified if you assume all the individual parameters are independent. $\endgroup$ Mar 20, 2020 at 9:18
  • $\begingroup$ @aaronsnoswell why does the independence of particles imply that the many interactions are approximated with a single average interaction? $\endgroup$
    – 900edges
    Dec 5, 2021 at 14:20
1
$\begingroup$

Mean-field approximation is a way to simplify the variational Bayes procedure. MFA makes it possible to use coordinate ascent to find the approximating function. See https://github.com/idnavid/misc/blob/master/variationalbayes_doc1.ipynb

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.