9
$\begingroup$

Let $X_1,\dots,X_n$ be an i.i.d. sample from the standard normal distribution. Let \begin{align} \mu_n = \mathbb{E}[X_{(n)} - X_{(n-1)}], \end{align} be the expected difference between the largest and second largest observations.

Question: Can we show that $\mu_n$ is decreasing in $n$?

In this answer it is shown that $\mu_n$ is asymptotically decreasing, i.e., it is decreasing for large enough $n$, and in this answer it is argued numerically that $\mu_n$ is also decreasing for finite $n$.

A good first step is to use a known order statistics formula (see e.g. this answer) to write \begin{align} \mu_n = n\int_{-\infty}^{\infty}\Phi^{n-1}(x)(1-\Phi(x))dx, \end{align} where $\Phi$ is the standard normal CDF. However, I am unable to show from this that $\mu_n$ is decreasing.

Addition: The intuition provided in a previously linked answer is that $\mu_n$ is decreasing if the $X_i$'s come from a thin-tailed distributions like the normal distribution, and that $\mu_n$ is constant or even increasing if the $X_i$'s come from a more heavy-tailed distribution (in particular, it is easy to show that $\mu_n$ is constant if the $X_i$'s come from the exponential distribution). Perhaps this intuition can help in showing that $\mu_n$ is decreasing for the normal distribution.

$\endgroup$
2
  • 1
    $\begingroup$ According to your definition, $\mu_n$ is a random variable. That is not consistent with your question, or with your integral. Do you mean its expected value? $\endgroup$
    – leonbloy
    Jan 5 at 15:30
  • $\begingroup$ @leonbloy Thank you for noticing. I have added the expectation. $\endgroup$
    – svonimir
    Jan 5 at 15:34

1 Answer 1

10
$\begingroup$

Let’s integrate the expression you provide for $\mu_n$ by parts to reduce the dependence on $n$:

\begin{eqnarray*} \mu_n &=& \int_{-\infty}^\infty n\Phi^{n-1}(1-\Phi)\mathrm dx \\ &=& \int_{-\infty}^\infty n\Phi^{n-1}\Phi'\frac{1-\Phi}{\Phi'}\mathrm dx \\ &=& \left[\Phi^n\frac{1-\Phi}{\Phi'} \right]_{-\infty}^\infty-\int_{-\infty}^\infty\Phi^n\left(\frac{1-\Phi}{\Phi'}\right)'\mathrm dx \\ &=& \int_{-\infty}^\infty\Phi^n\left(1+\frac{\Phi''(1-\Phi)}{\Phi'^2}\right)\mathrm dx\;,\end{eqnarray*}

where the boundary term vanishes for a normal distribution. Since this expression can be extended to real $n$, we can differentiate it with respect to $n$, yielding $$ \frac{\partial\mu_n}{\partial n}=\int_{-\infty}^\infty\Phi^n\log\Phi\left(1+\frac{\Phi''(1-\Phi)}{\Phi'^2}\right)\mathrm dx\;. $$

So far we only assumed a normal distribution in assuming $\Phi'\ne0$ and in concluding that the boundary term vanishes, so this result holds for all distributions that fulfil those conditions (possibly with different limits on the integral). It also holds if the boundary term is non-zero but doesn’t depend on $n$. For instance, for an exponential distribution, with

$$ \Phi=1-\mathrm e^{-\lambda x}\;,\\ \Phi'=\lambda\mathrm e^{-\lambda x}\;,\\ \Phi''=-\lambda^2\mathrm e^{-\lambda x}\;, $$

the boundary term is $1$, independent of $n$, and we have

\begin{eqnarray*} 1+\frac{\Phi''(1-\Phi)}{\Phi'^2} &=& 1+\frac{-\lambda^2\mathrm e^{-\lambda x}\left(1-\left(1-\mathrm e^{-\lambda x}\right)\right)}{\left(\lambda\mathrm e^{-\lambda x}\right)^2} \\ &=& 0\;, \end{eqnarray*}

confirming that $\mu_n$ is constant for this type of distribution.

Since $\Phi^n\log\Phi$ is negative and the remaining factor doesn’t depend on $n$, $\mu_n$ is strictly increasing or strictly decreasing if this factor is everywhere negative or everywhere positive, respectively. If the factor changes sign, then $\frac{\partial\mu_n}{\partial n}$ may change sign as $\Phi^n$ gets increasingly concentrated near $\Phi=1$ for increasing $n$.

For a uniform distribution, the boundary term vanishes, and we have $\Phi''=0$, so $\mu_n$ is strictly decreasing, as expected.

For the standard normal distribution, we have

\begin{eqnarray*} \Phi &=&\frac12\left(1+\operatorname{erf}\left(\frac x{\sqrt2}\right)\right)\;,\\ \Phi' &=& \frac1{\sqrt{2\pi}}\exp\left(-\frac{x^2}2\right)\;,\\ \Phi'' &=& -\frac1{\sqrt{2\pi}}x\exp\left(-\frac{x^2}2\right)\;, \end{eqnarray*}

so

\begin{eqnarray*} 1+\frac{\Phi''(1-\Phi)}{\Phi'^2} &=& 1-\sqrt{\frac\pi2}x\exp\left(\frac{x^2}2\right)\left(1-\operatorname{erf}\left(\frac x{\sqrt2}\right)\right)\;. \end{eqnarray*}

With

\begin{eqnarray*} \sqrt{\frac\pi2}\left(1-\operatorname{erf}\left(\frac x{\sqrt2}\right)\right) &=& \sqrt2\int_{x/\sqrt2}^\infty\exp\left(-t^2\right)\mathrm dt \\ &=& \int_x^\infty\exp\left(-\frac{u^2}2\right)\mathrm du \\ &=& \int_x^\infty\left(-\frac1u\right)\left(-u\exp\left(-\frac{u^2}2\right)\right)\mathrm du \\ &=& \left[-\frac1u\exp\left(-\frac{u^2}2\right)\right]_x^\infty-\int_x^\infty\frac1{u^2}\exp\left(-\frac{u^2}2\right)\mathrm du \\ &=& \frac1x\exp\left(-\frac{x^2}2\right)-\int_x^\infty\frac1{u^2}\exp\left(-\frac{u^2}2\right)\mathrm du\;, \end{eqnarray*}

this becomes

\begin{eqnarray*} 1+\frac{\Phi''(1-\Phi)}{\Phi'^2} &=& 1-x\exp\left(\frac{x^2}2\right)\left(\frac1x\exp\left(-\frac{x^2}2\right)-\int_x^\infty\frac1{u^2}\exp\left(-\frac{u^2}2\right)\mathrm du\right) \\ &=& x\exp\left(\frac{x^2}2\right)\int_x^\infty\frac1{u^2}\exp\left(-\frac{u^2}2\right)\mathrm du\ \\ &\gt&0\;, \end{eqnarray*}

so indeed $\mu_n$ is strictly decreasing for the standard normal distribution.

$\endgroup$
2
  • 1
    $\begingroup$ The last formula concerns the derivative of the Inverse Mills ratio: $$(\text{Mill})' = \frac{\Phi''(1-\Phi) +(\Phi')^2}{(1-\Phi)^2}$$ And the Inverse Mills ratio is increasing. $\endgroup$
    – NN2
    Jan 5 at 21:51
  • 1
    $\begingroup$ @NN2: Thanks, that's interesting! I've also added a proof of the positivity of this factor for a normal distribution by direct calculation. (It should be pointed out for the benefit of other readers that the linked proof that the inverse Mills ratio is increasing assumes a normal distribution.) $\endgroup$
    – joriki
    Jan 5 at 22:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .