Is the Sample Median Biased?

Surrounding the topic of the median, I have heard the following statements being made informally :

• Unlike the mean, the median is more "robust" (i.e. less sensitive to outliers)
• But on the other hand, the median in the general case is usually "biased" whereas the mean is not.
• However, both the mean and the median tend to be "consistent"

I am trying to understand these statements from a mathematical perspective.

I found the following question (Prove that the sample median is an unbiased estimator) which shows that for the Normal Distribution, the sample median is actually an unbiased estimator for the true population mean - but this is not what I am interested in showing.

• I have tried to find a mathematical proof online that shows the sample median is not always equal to the population median - but I could not find such a proof.
• I also could not find a direct mathematical proof that shows the sample median is consistent (i.e. becomes closer and closer to the population median as the number of data points increases).

Can someone please show me how to prove these?

Thanks!

Notes:

• This belongs on CrossValidated.SE. Aug 13, 2023 at 0:49
• Do you mean something like Simpson's paradox? en.wikipedia.org/wiki/Simpson%27s_paradox Aug 13, 2023 at 2:14
• Just something very general... e.g. can we prove that the expected value of the sample median is not always equal to the population median? Aug 13, 2023 at 12:59
• Sample median is unbiased for population median for symmetric distributions, as discussed in the linked post. So consider a skewed distribution, like the Exponential. Suppose you draw a sample of size $3$ from $\text{Exp}(\theta)$. One can easily check that the sample median here is biased for the population median. Aug 13, 2023 at 15:37
• @StubbornAtom : thank you so much for your reply! in general, is there a way to "measure the symmetry" of a distribution? e.g. suppose i have some random variable X with probability distribution p(x). How can I tell is p(x) is symmetric? thank you so much! Aug 13, 2023 at 15:39

Let $$X(n)$$ be a sample of size $$n$$ from a continuous distribution $$P(x)$$ with median $$m$$.

The empirical distribution of $$X(n)$$ is $$F_n$$ (monotonic increasing piecewise constant).

From Glivenko-Cantelli theorem $$F_n \to F$$ uniformly, which implies that the median of the $$F_n$$ will converge to $$m$$ almost surely.

Proof

The Glivenko-Cantelli theorem states that $$\sup_{x \in \mathbb{R}}\left| F(x) - F_n(x)\right| \xrightarrow{a.s.} 0$$. We know that $$0 \leq \left| F(m) - F_n(m)\right| \leq \sup_{x \in \mathbb{R}}\left| F(x) - F_n(x)\right|$$ by definition of the supremum and absolute value. Therefore,

$$0 \leq \left| F(m) - F_n(m)\right| = \left| \frac12 - F_n(m)\right| \leq \sup_{x \in \mathbb{R}}\left| F(x) - F_n(x)\right| \xrightarrow{a.s.} 0$$ $$\implies \left| \frac12 - F_n(m)\right| \xrightarrow{a.s.} 0$$

Notice we made no distributional assumptions about $$P$$ except continuity.

Note: Need to add a little more nuance in the cases where $$P$$ has atoms (jumps) in it's CDF.

In the case we can define the median of discrete distribution $$P$$ as the value(s) where the CDF $$F(x)$$ intersects the line $$F=\frac12$$. This means that we can define $$m$$ as follows:

$$m(P):= \arg \min_{x} F(x)\geq \frac12$$

This definition conveniently applies to both discrete, mixed, and continuous distributions.

Due to the monotonicity of the $$F(x)$$, we will have a unique value for $$m$$ except in the case that @kimchilover points out (we have an interval of values where $$F(x)=\frac12$$.

In that case, the sample median will converge in probability to a set:

Let $$m_i(P)$$ be the sample median of size $$i$$ drawn from $$P$$ with CDF $$F$$. Let $$M$$ be the set

$$M:=\left\{ x: F(x) \leq \frac12\right\} \cap \left\{ x:F(x) \geq \frac12 \right\}$$

Then the sequence of sample medians achieves the following limit:

$$\lim_{n \to \infty} P\left( m_i \in \{\inf M-\epsilon\} \cup \{\sup M+\epsilon\}\right) = 1\;\forall \epsilon > 0$$

In other words, the infinite sequence of $$m_i(P)$$ will fall in the above set for all but a finite number of times. This is because the order statistics of the samples will "cluster" around each endpoint of the interval, so that you often "flip flop" between different endpoints as the next sample values falls to the left or right of the interval.

The above generalizes the earlier definitions of median:

Note that when the median is unique (e.g., a continuous distribution or we have a "jump" in the CDF from $$<\frac12$$ to $$>\frac12$$ where the location of the jump defines a unique median) then the above reduces to saying the sample median converges in probability to the unique median. This is weaker than almost sure convergence from Glivenko-Cantelli, but for all practical purposes is just as good.

• What if $F(x)=1/2$ for all $x$ in an interval centered at $m$? Aug 15, 2023 at 22:19
• @Annika: thank you so much for your answer! So the sample medoids is consistent but not unbiased? Aug 15, 2023 at 22:35
• In your 4th paragraph... : the glivenko cantelli theorem states....we know that 0.. i think you forgot something in the inequality? Aug 15, 2023 at 22:36
• The sample mean does not always exist... when is this the case? Aug 15, 2023 at 22:37
• @Annika Good! I was bothered by your previous claim that there was no distributional condition on $F$ to ensure the result. Aug 16, 2023 at 1:35