17
$\begingroup$

In lots of Bayesian papers, people use variational approximation. In lots of them they call it "mean-field variational approximation". Does anyone know what is the meaning of mean-field in this context?

$\endgroup$

3 Answers 3

14
$\begingroup$

I found some intuitions that might answer this; based on the definition of "mean-field" at Wikipedia, mean field theory (MFT also known as self-consistent field theory) studies the behaviour of large and complex stochastic models by studying a simpler model. Such models consider a large large number of small interacting individuals who interact with each other. The effect of all the other individuals on any given individual is approximated by a single averaged effect, thus reducing a many-body problem to a one-body problem.

So basically approximating the inference and learning problem, using independence assumptions and decomposition into several products, brings the notion of "mean-field" approximation.

$\endgroup$
7
$\begingroup$

I believe the mean-field approximation used in mean-field variational Bayes is the assumption that the posterior approximation factorizes over the parameters

$$q(\mathbf{\theta}) = q_1(\theta_1) q_2(\theta_2) \dots q_n(\theta_n)$$

$\endgroup$
3
  • 2
    $\begingroup$ Is it obvious why this factorization is called "mean-field"? $\endgroup$
    – Daniel
    Commented Apr 14, 2017 at 15:08
  • 1
    $\begingroup$ It gets this name from it's origin in Physics. In a system with many components that interact, the many interactions are approximated with a single average interaction - i.e. the mean field. The analogy to Bayesian Variational Inference is that a complicated posterior can be simplified if you assume all the individual parameters are independent. $\endgroup$ Commented Mar 20, 2020 at 9:18
  • $\begingroup$ @aaronsnoswell why does the independence of particles imply that the many interactions are approximated with a single average interaction? $\endgroup$
    – 900edges
    Commented Dec 5, 2021 at 14:20
3
$\begingroup$

Mean-field approximation is a way to simplify the variational Bayes procedure. MFA makes it possible to use coordinate ascent to find the approximating function. See https://github.com/idnavid/misc/blob/master/variationalbayes_doc1.ipynb

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .