# Understanding some basics

Let $x_1,\ldots,x_n$ be any numbers and $\bar{x} = (x_1+ \ldots + x_n)/n$. Then I need to prove that $$min_a \sum_{i=1}^n (x_i - a)^2 = \sum_{i=1}^n (x_i - \bar{x})^2$$

I do not need the answer but I need an visual/intuitive understanding of the question. I do not understand what min$_a$ mean. Actually I'm kind of struggling to understand "minimization concepts" i.e. the value $a$ is not defined or anything so what does minimization of $a$ mean ? It would be great if someone can explain the concept in general too. Thanks.

I got the answer perfectly for what I asked. I tried to solve the problem as well. I got stuck but here is my try.

$$\sum_{i=1}^n (x_i - a)^2 = \sum_{i=1}^n (x_i - \bar{x} + \bar{x} - a)^2$$ $$= \sum_{i=1}^n (x_i - \bar{x})^2 + 2 \sum_{i=1}^n (x_i - \bar{x})(x_i - a) + \sum_{i=1}^n(x_i - a)^2$$ ...

Can anyone help me after this ? Thanks again.

-
Kind of trying to get the answer with Yuval's explanation. If I'm not wrong $a$ should be minimized at $\bar{x}$ but how to prove it ? – Sunil Mar 5 '11 at 6:46

Given $x_1,\ldots,x_n$, you can define a function $$f(a) = \sum_{i=1}^n (x_i-a)^2.$$ This function is defined for all real $a$, and in some sense it measures the centrality of $a$.

You can try some simple example on Wolfram alpha, say "plot((a-1)^2+(a-2)^2+(a-4)^2,a=0..5)". In this case the data points are $1,2,4$, and you get a nice, parabolic-like shape (this is no coincidence). The lowest point of the function is the minimum.

The function $f(a)$ is bounded from below (since $f(a) \geq 0$), and so it cannot happen that $f(a)$ can be arbitrarily small. Therefore there must be a value below which $f(a)$ cannot reach, and moreover an optimal one - this is known as the infimum. The infimum is a value $L$ such that $f(a) \geq L$ always, and moreover for each $l > L$, there is some value of $a$ such that $f(a) < l$; that expresses the fact that $L$ is optimal.

In general, it might be that $f(a)$ can get arbitrarily close to the infimum, but never reach it. In the case at hand, this doesn't happen, and the function $f(a)$ is actually minimized at some point, namely the average (or mean).

The equation you state can be used to justify the definition of average - the average is the value that minimizes the average squared distance from the datapoints. You could choose other criteria - for example, if you replace squared distance by absolute distance, then the optimal value of $a$ is the median.

The reason that people care about squared distance is that it's much easier to work with, and the resulting theory is very nice. For example, the central limit theorem states that in many cases, processes converge to a normal distribution whose parameters depend on the mean and variance of the original distribution - the variance is just $\min f(a)$ (normalized).

-
Very clear and a perfect answer. Thank you very much. – Sunil Mar 5 '11 at 6:36

The question is about minimization with respect to a variable a. For this we use the derivative: the minimum of the function $f(a)$ is where the first derivative is zero and the second is positive.
Now write $$f(a) = \sum_{i=1}^n(x_i-a)^2 = \sum_{i=1}^n a^2-2ax_i+x_i^2$$ Then the derivative is $$f'(a) = \sum_{i=1}^n (2a - 2x_i) = 2(an - \sum_{i=1}^n x_i)$$ We see: if $a*n$ equals the sum we have $f'(a)$ a zero. So we can compute $a_0$ by $$a_0 = \frac{\sum_{i=1}^n x_i }n$$ Next, if the second derivative is positive, then indeed $f(a_0)$ is also a minimum: $$f''(a) = 2n$$ and this is positive - independent of a and/or the $x_i$ . But the formula for $a_0$ is just the definition of the mean: $x_{mean} =\frac{\sum_{i=1}^n x_i }n$

-