# Intuition on Minimizing Squared Distance [duplicate]

Say that we have a bunch of random-values $$x_1, x_2, ..., x_n$$

Let $$\bar{X}$$ be the mean of these random-values. The variance of this sample-set is given by:

$$\sum(x_i-\bar{X})^2$$

Apparently, the above sum is smaller than if $$\bar{X}$$ were any other value besides the average of the $$n$$ observations.

I've seen a Calculus-based explanation...are there other ways to show this? Perhaps geometrically or algebraically?

Thanks.

• If by "bigger" you mean "smaller" and by "maximizing" you mean "minimizing", then yes. Nov 29, 2019 at 17:19
• Lol - thanks @DavidK Fixed it Nov 29, 2019 at 17:55

Let $$a=\bar{X}$$.

Our goal is to show that for $$b\in\mathbb{R}$$, the sum $$\sum (x_i-b)^2$$ is minimized when and only when $$b=a$$.

Solution #$$1$$:$$\;$$Using only elementary algebra . . . \begin{align*} &\sum(x_i-b)^2-\sum(x_i-a)^2\\[4pt] =\;& \left(\sum x_i^2-2b\sum x_i+nb^2\right) - \left(\sum x_i^2-2a\sum x_i+na^2\right) \\[4pt] =\;& n(b^2-a^2)-2(b-a)\sum x_i \\[4pt] =\;& n(b^2-a^2)-2(b-a)(na) \\[4pt] =\;& n(b^2-2ab+a^2) \\[4pt] =\;& n(b-a)^2 \\[4pt] \ge\;&\,0 \\[4pt] \end{align*} with equality if and only if $$b=a$$.

Solution #$$2$$:$$\;$$Using vector geometry . . .

In $$\mathbb{R}^n$$, let \begin{align*} P&=(x_1,...,x_n)\\[4pt] A&=(a,...,a)\\[4pt] B&=(b,...,b)\\[4pt] \end{align*} and let $$l$$ be the line with parametric form $$\begin{cases} x_1=t\\ \;\vdots\;\;\;\,\vdots\;\;\vdots\\ x_n=t \end{cases}$$ Interpreted geometrically,

• $$\sum (x_i-a)^2$$ is the square of the distance from $$P$$ to $$A$$.$$\\[4pt]$$
• $$\sum (x_i-b)^2$$ is the square of the distance from $$P$$ to $$B$$.

Note that the point $$A$$ is on the line $$l$$, and $$B$$ is just an arbitrary point on $$l$$.

Thus, with our geometric interpretation, our goal is to show that $$A$$ is the unique point on $$l$$ which is closest to $$P$$.

It suffices to show $$\vec{PA}\perp l$$.

Letting $$\vec{U}=\langle{1,...,1}\rangle$$, we get \begin{align*} &\vec{PA}\cdot\vec{U}\\[4pt] =\;&\left(\vec{A}-\vec{P}\right)\cdot \vec{U}\\[4pt] =\;&\vec{A}\cdot\vec{U}-\vec{P}\cdot\vec{U}\\[4pt] =\;&na-\sum x_i\\[4pt] =\;&0\\[4pt] \end{align*} so $$\vec{PA}\perp l$$, as required.