# Trying to understand the math in a neuroscience article by Karl Friston

I am trying to understand a neuroscience article by Karl Friston. In it he gives three equations that are, as I understand him, equivalent or inter-convertertable and refer to both physical and Shannon entropy. They appear as equation (5) in the article at http://www.fil.ion.ucl.ac.uk/spm/doc/papers/Action_and_behavior_A_free-energy_formulation.pdf (DOI 10.1007/s00422-010-0364-z). Here they are:

• Energy minus entropy
• Divergence plus surprise
• Complexity minus accuracy

\begin{align*}F &=−\left\langle \ln p (\tilde{s},\Psi |m)\right\rangle_q+\left\langle \ln q(Ψ|\mu)\right\rangle_q \\ &= D\left(q(\Psi|\mu)\ ||\ p(\Psi|\tilde{s},m)\right)−\ln p(\tilde{s}|m) \\ &= D\left(q(\Psi|\mu)\ ||\ p(\Psi|m)\right)−\left\langle \ln p(\tilde{s}|\Psi,m)\right\rangle_q \end{align*}

The things I am struggling with at this point are 1) the meaning of the || in the 2nd and 3rd versions of the equations, 2) the negative logs. Any help in understanding how each of these equations amounts to what Fristen describes it to be (at the left of the equation) would be greatly appreciated. For example, in the 1st equation, in what sense are the terms energy and entropy, etc? Is the entropy, Shannon or thermodynamic or both?

• I have edited your post to match the exact presentation of this portion of the paper. I cannot answer your question, however. Jul 12, 2014 at 16:34
• @Arkamis, thanks for your edits and the references, which are helpful. Jul 12, 2014 at 21:17
• Cross-posted on Data Science, Cross Validated, Cognitive Sciences
– honi
Nov 17, 2015 at 19:05

Here, $\langle \cdot \rangle_q$ means the expectation or mean under the density $q$ and $D(\cdot\ ||\ \cdot)$ is the cross-entropy or Kullback-Leibler divergence between the two densities.