Intuition on Minimizing Squared Distance [duplicate]

Question

Say that we have a bunch of random-values $x_1, x_2, ..., x_n$

Let $\bar{X}$ be the mean of these random-values. The variance of this sample-set is given by:

$$\sum(x_i-\bar{X})^2$$

Apparently, the above sum is smaller than if $\bar{X}$ were any other value besides the average of the $n$ observations.

I've seen a Calculus-based explanation...are there other ways to show this? Perhaps geometrically or algebraically?

Thanks.

---

Answer 1

Let $a=\bar{X}$.

Our goal is to show that for $b\in\mathbb{R}$, the sum $$\sum (x_i-b)^2$$ is minimized when and only when $b=a$.

Solution #$1$:$\;$Using only elementary algebra . . . \begin{align*} &\sum(x_i-b)^2-\sum(x_i-a)^2\\[4pt] =\;& \left(\sum x_i^2-2b\sum x_i+nb^2\right) - \left(\sum x_i^2-2a\sum x_i+na^2\right) \\[4pt] =\;& n(b^2-a^2)-2(b-a)\sum x_i \\[4pt] =\;& n(b^2-a^2)-2(b-a)(na) \\[4pt] =\;& n(b^2-2ab+a^2) \\[4pt] =\;& n(b-a)^2 \\[4pt] \ge\;&\,0 \\[4pt] \end{align*} with equality if and only if $b=a$.

Solution #$2$:$\;$Using vector geometry . . .

In $\mathbb{R}^n$, let \begin{align*} P&=(x_1,...,x_n)\\[4pt] A&=(a,...,a)\\[4pt] B&=(b,...,b)\\[4pt] \end{align*} and let $l$ be the line with parametric form $$ \begin{cases} x_1=t\\ \;\vdots\;\;\;\,\vdots\;\;\vdots\\ x_n=t \end{cases} $$ Interpreted geometrically,

• $\sum (x_i-a)^2$ is the square of the distance from $P$ to $A$.$\\[4pt]$
• $\sum (x_i-b)^2$ is the square of the distance from $P$ to $B$.

Note that the point $A$ is on the line $l$, and $B$ is just an arbitrary point on $l$.

Thus, with our geometric interpretation, our goal is to show that $A$ is the unique point on $l$ which is closest to $P$.

It suffices to show $\vec{PA}\perp l$.

Letting $\vec{U}=\langle{1,...,1}\rangle$, we get \begin{align*} &\vec{PA}\cdot\vec{U}\\[4pt] =\;&\left(\vec{A}-\vec{P}\right)\cdot \vec{U}\\[4pt] =\;&\vec{A}\cdot\vec{U}-\vec{P}\cdot\vec{U}\\[4pt] =\;&na-\sum x_i\\[4pt] =\;&0\\[4pt] \end{align*} so $\vec{PA}\perp l$, as required.