This is from a statistical physics problem, but it is the mathematics behind it that I am stuck on here:

Consider a large number $N$ of distinguishable particles distributed among $M$ boxes.

We know that the total number of possible microstates is $$\Omega=M^N$$ and that the number of microstates with a distribution among the boxes given by the configuration $[n_1, n_2, ..., n_M]$ is given by $$\frac{N!}{\prod_{j=1}^M (n_j)!}\tag{1}$$ In the most likely configuration there are $$n_0 = \frac{N}{M}$$ particles in each box. Let $\Omega_0$ denote the statistical weight of this configuration and $p_0$ its probability. Now consider moving $\delta n$ particles from box $1$ to box $2$, giving the new configuration $$[n_0 − δn, n_0 + δn, n_0, n_0, ..., n_0]$$ Show that $$\fbox{$\color{blue}{\delta\left(\ln\Omega_{\{n\}}\right)\approx -\delta n\ln (n_0)-\frac{(\delta n)^2}{2 n_0}}$}$$ (Hint: you should use Stirling’s approximation here)

The image below shows the situation: Two boxes

Here is the answer as written by the author: Two boxes

This might be a bit hard to read so I have typed it out word for word below:

$$\ln\Omega_{\{n\}}=\ln(N!)-\sum_j\ln (n_j!)$$ The change in $\ln\Omega$ due to a change $\delta n$ in ONE box is $$\delta(\ln\Omega_{\{n\}})=-\ln([n_0+\delta_n]!)+\ln(n_0!)$$ Taylor expand $\ln (n!)$ $\color{red}{\text{(To second 2nd order since we are already at a maximum for}}$ $\color{red}{\ln\Omega_{\{n\}}}$$\color{red}{)}$ using Stirling: $$\ln (n!)\approx \underbrace{n_0\ln (n_0)-n_0}_{0th}+\underbrace{\ln (n_0)\cdot\delta_n}_{1st}+\underbrace{\frac{(\delta n)^2}{2n_0}}_{2nd}+\cdots$$ $$\implies \delta(\ln\Omega_{\{n\}})=-\ln (n_0)\cdot\delta n-\frac{(\delta n)^2}{2n_0}$$

The red bracket can be ignored as it was just used in the previous part of the question where we had to show that $$n_0=\frac{N}{M}$$ is the most likely configuration which we did my differentiating and setting to zero to maximize.

Finally I get to my question:

I understand everything up to the point where it says "Taylor expand $\ln n!$", firstly where on earth is $n$ even defined? I see $n_0$ and $N$ but not $n$.

I can only assume that the author meant 'Taylor expand $\ln([n_0+\delta_n]!)$' since I fail to see how that expression could ever get the factors $n_0$ and $\delta_n$.

I know that the general Taylor expansion formula is given by $$f(a)+\frac {f'(a)}{1!} (x-a)+ \frac{f''(a)}{2!} (x-a)^2+\frac{f^{(3)}(a)}{3!}(x-a)^3+ \cdots$$

and that Stirlings approximation is as follows $$\ln(k!)\approx k\ln k - k +\frac12\ln k$$

I have used both these formulae, but not both together.

I'm very confused about how to proceed with this, so I naively apply Stirlings approximation first:

$$\ln([n+\delta n]!)\approx (n+\delta n)\ln(n+\delta n)-(n+\delta n)+\frac12\ln(n+\delta n)$$

But now I am completely stuck. Could someone please explain how the author was able to reach the result?


A comment below raised an important point. So I will just mention that for the purposes of this question the statistical weight $\Omega$ is equal to the number of microstates.


I must apologize there was a contradiction in my question due to a typo.

What I needed to show was that $$\fbox{$\color{blue}{\delta\left(\ln\Omega_{\{n\}}\right)\approx -\delta n\ln (n_0)-\frac{(\delta n^2)}{2 n_0}}$}$$ and this is given in blue in the first quotation box, very sorry about this.


I acknowledge that those of you that answered actually figured out that there must be a similar expression when considering both boxes. So as promised here is part 2 of the question as written by the author: part 2 of the question

I'll type it word for word again anyway just in case it's hard to read:

For a change to TWO boxes, moving $\delta n$ from $1$ to $2$, $$\delta\left(\ln\Omega_{\{n\}}\right)=-\ln (n_0)\cdot(-\delta n)-\frac{\left(-\delta n\right)^2}{2n_0}$$ $$\qquad\qquad\qquad\qquad\qquad-\ln (n_0)\cdot(+\delta n)-\frac{\left(+\delta n\right)^2}{2n_0}=-\frac{\delta n^2}{n_0}$$ $$\color{red}{\Huge{\star}}\quad\ln\Omega_{\{n\}}=\ln\Omega_0-\frac{\delta n^2}{n_0}$$

Many thanks.

  • 1
    $\begingroup$ how about finding a Taylor expansion for ln(z) first? maybe with a = 1 $\endgroup$
    – Cato
    Jan 10, 2017 at 11:50
  • $\begingroup$ or $log(x + y) = log(x) + y/x - y^2/2x....$ I think if you use that after first collecting your ln(x+y) terms $\endgroup$
    – Cato
    Jan 10, 2017 at 12:11
  • 1
    $\begingroup$ There's a mistake there, you have $\log(x+y)=\log(x)+y/x-y^2/2x^2+\dots$ when $0<|y|<x$. This is just Taylor expansion. That said, this is not exactly Taylor expansion, because $n=n_0+\delta n$ is not really a continuous variable. But you can proceed by applying Stirling and then "pretending" $n_0+\delta n$ is a continuous variable to get an asymptotic. A bit more rigorously you might choose to explain things through the Gamma function (which if I recall correctly satisfies Stirling even at large positive real values). $\endgroup$
    – Ian
    Jan 13, 2017 at 12:19
  • 2
    $\begingroup$ How are you defining statistical weight? How is it related to the number of microstates $\Omega_{\{n\}}$ for example? $\endgroup$
    – nio
    Jan 14, 2017 at 6:37
  • 2
    $\begingroup$ @NafizIshtiaqueOrnob Oh sorry, I have misinterpreted your question. The statistical weight $\Omega_{\{n\}}$ is equal to the number of microstates. Which is basically what you said in your comment. I will edit my question to reflect this. Thanks. $\endgroup$
    – BLAZE
    Jan 14, 2017 at 10:18

2 Answers 2


We assume the Stirling approximation (first order) $$\log(n!)\approx n \log(n) - n = g(n) \tag{1}$$

And the Taylor expansion:

$$g(x)\approx g(x_0)+g'(x_0)(x-x_0)+g''(x_0)\frac{(x-x_0)^2}{2}=\\ =x_0\log(x_0)-x_0 + \log(x_0)\delta+\frac{\delta^2}{2 x_0} \tag{2}$$

where $\delta = x-x_0$ (increment).

Then the change in the box that decrements its counting (box 1) by $\delta$ is

$$\Delta_1(\ln\Omega_{\{n\}})=-\ln((n_0-\delta)!)+\ln(n_0!)=\\= \log(n_0)\delta-\frac{\delta^2}{2 n_0} \tag{3}$$

The change in the other box that increments its counting (box 2) by $\delta$ is

$$\Delta_2(\ln\Omega_{\{n\}})=-\ln((n_0+\delta)!)+\ln(n_0!)=\\ =-\log(n_0)\delta-\frac{\delta^2}{2 n_0} \tag{4}$$

The total change is then $$\Delta(\ln\Omega_{\{n\}})= \Delta_1(\ln\Omega_{\{n\}})+\Delta_2(\ln\Omega_{\{n\}})=-\frac{\delta^2}{n_0} \tag{5}$$

(that's why we need to take a second-order Taylor expansion: because the linear terms cancel - not because "we are at a local maximum" - actually the result is valid for any $n_0$, not only for $n_0=N/M$)

The original statement $$\delta\left(\ln\Omega_{\{n\}}\right)\approx \ln\Omega_0-\frac{\delta n^2}{2 n_0}$$ seems to have two errors: First, if mixes the value of the statistical weight with its change. If we are speaking of the change, then the first "$\delta$" is right, but the initial term $\ln\Omega_0$ is wrong - sanity check: if $\delta n=0$ the change should be zero). Furthermore, the result seems to have a missing two factor (indeed, the copied derivation emphasizes that the first variation is due to one box, but then it seems ot forget about the other one).

Edit: Regarding your edit: your "equation in blue" is only correct if it's understood to be the change in one box (see eq $3$). If it's understood (as it seems) as the total change, then it's wrong. The net change is given by eq. $(5)$

  • $\begingroup$ Thank you for taking the time to write an excellent answer (+1). Please see the comment below Nafiz Ishtiaque Ornob's answer. $\endgroup$
    – BLAZE
    Jan 14, 2017 at 22:25
  • $\begingroup$ There was a typo causing a contradiction in my question. It's fixed now, so it will make sense. Sorry about this. $\endgroup$
    – BLAZE
    Jan 14, 2017 at 23:14
  • $\begingroup$ Hi I have now uploaded the second part of the question. Is it still consistent with your answer? $\endgroup$
    – BLAZE
    Jan 19, 2017 at 17:11
  • $\begingroup$ @BLAZE As you can easily check, my answer totally agrees with you last update. It would had been nice if you had posted the whole question on one shot. $\endgroup$
    – leonbloy
    Jan 19, 2017 at 18:05
  • $\begingroup$ Congratulations on winning the bounty, it was a tough a choice as both answers were very good. Yes you're right I should have posted the whole question in one shot. Many thanks. $\endgroup$
    – BLAZE
    Jan 20, 2017 at 8:26

Let us define the following two configurations:

  • An "optimal" configuration: $\mathbf n_1 := [n_0, \cdots, n_0]$ .
  • The perturbed configuration: $\mathbf n_2 := [n_0 - \delta n, n_0 + \delta n, n_0, \cdots, n_0]$ .

Statistical weights of the two configurations are: $$ \Omega_0 := \Omega_{\{\mathbf n_1\}} = \frac{N!}{(n_0!)^M}\,, \qquad \Omega_{\{\mathbf n_2\}} = \frac{N!}{(n_0-\delta n)!(n_0 + \delta n)! (n_0!)^{M-2}} \,. $$ Taking log of these two weights we get: \begin{align} \ln \Omega_0 =&\; \ln N! - M \ln n_0!\,, \\ \qquad \ln \Omega_{\{\mathbf n_2\}} =&\; \ln N! - \ln (n_0-\delta n)! - \ln(n_0+\delta n)! - (M-2) \ln n_0!\,. \end{align} Log of the number of microstates is proportional to entropy, so let's call these two quantities the entropies of the two configurations (ignoring the proportionality constant, which is called the Boltzmann constant). Change in entropy due to perturbation is: $$\ln \Omega_{\{\mathbf n_2\}} - \ln \Omega_0 = - \ln(n_0-\delta n)! - \ln(n_0 + \delta n)! + 2\ln n_0!\,. \tag{1}$$

Assuming $n_0$ and $n_0-\delta n$ are both fairly large we use Stirling's approximation, nameley $\ln m! \approx m \ln m - m$ for large $m$, to approximate (1): $$ \ln \Omega_{\{\mathbf n_2\}} - \ln \Omega_0 \approx -(n_0 - \delta n) \ln (n_0 - \delta n)-(n_0 + \delta n) \ln (n_0 + \delta n) + 2 n_0 \ln n_0\,. \tag{2}$$ Now we use Taylor expansion to expand $\ln(n_0 \pm \delta n)$ by treating $\delta n$ as small compared to $n_0$. The expansion of $\ln(x+\epsilon)$ around $x$ is given by: $$ \ln(x + \epsilon) = \ln x + \frac{\epsilon}{x} - \frac{\epsilon^2}{2x^2} + \mathcal O\left(\left(\frac{\epsilon}{x}\right)^3\right)\,.$$ Using this in (2), and keeping terms upto quadratic order in $\delta n$ we find: $$ \ln \Omega_{\{\mathbf n_2\}} - \ln \Omega_0 \approx -\frac{(\delta n)^2}{n_0}\,.$$ So the entropy of the perturbed system is given by: $$ \ln \Omega_{\{\mathbf n_2\}} \approx \ln\Omega_0 -\frac{(\delta n)^2}{n_0} \,. \tag{3}$$

The reason why in (3) there is no term linear in $\delta n$ is very physical, an optimal configuration has the highest entropy, therefore change in entropy as you perturb from an optimal configuration must begin at least at the quadratic order.

  • $\begingroup$ Thank you very much for a fine answer indeed. Your time and effort in writing this answer is greatly appreciated. Firstly, in your answer you have stumbled across the second part of the question which involves the second box. I shall add the solution of this to my original question. Then we might be able to understand what the author was referring to when he wrote that expression for the first part. I omitted the second part as it was not needed to get the first part. But I will add it now anyway. Thanks again (+1) :-) $\endgroup$
    – BLAZE
    Jan 14, 2017 at 22:22
  • $\begingroup$ Your answer will make much more sense now as much to my shame there was a typo causing a contradiction in my question. It's fixed now, so it will make sense. Sorry. $\endgroup$
    – BLAZE
    Jan 14, 2017 at 23:13
  • $\begingroup$ @BLAZE You're welcome! typos happen all the time, no need to worry about it :) Btw, the first blue equation should probably not have the 2 in the second term if it's meant to be the entropy of the configuration $[n_0-\delta n, n_0 + \delta n, n_0, \cdots, n_0]$. If it's meant to be the change in entropy then it should only be $-\frac{(\delta n)^2}{n_0}$, without the first term. $\endgroup$
    – nio
    Jan 14, 2017 at 23:34
  • $\begingroup$ Hi, thanks. The $-\dfrac{(\delta n)^2}{n_0}$ term is a correct term to the answer for the second part of the question when we consider both the boxes (which I will be adding soon). But for now all I can tell you is that the $2$ must be there. I have checked the rest of the question very carefully, and the solution when considering just one box definitely has a $2$ in the denominator. Best regards. $\endgroup$
    – BLAZE
    Jan 15, 2017 at 0:03
  • 1
    $\begingroup$ @BLAZE Great, I just removed the "Confusion"s since there are no more confusions about the question and the provided solution :) $\endgroup$
    – nio
    Jan 19, 2017 at 19:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .