# Finding the minimum for this formula

I think if we want to calculate $\min_x \sum_i (b_i - x)^2$, the answer should be the mean of $b_i$, right? Now if we add a weight to each term and make it $\min_x \sum_i w_i(b_i - x)^2$, what's the closed form solution for $x$ then?

-
Differentiate, set to zero, solve for $x$. Which part is unfamiliar? –  Gerry Myerson Dec 13 '11 at 0:36
@GerryMyerson: Yeap...turns out my brain stuck at that point...seems a trivial problem... –  derekhh Dec 13 '11 at 0:39
At which point? The differentiation? the setting equal to zero? the solving for $x$? –  Gerry Myerson Dec 13 '11 at 2:06
@GerryMyerson : There's no denying that what you propose will solve the problem. I took a course in graduate school where we did a gillion exercises that included generalizations of this involving matrices, in which the (generalization of) the method you suggest would have been a nightmare. Various things involving designs of experiments in which there were lots of Wishart matrices. (All this reminds me that there's a lot of stuff not covered in Wikipedia. There's still no "expected mean squares article.) So I am partial to completing the square. –  Michael Hardy Dec 13 '11 at 2:10
It's just the weighted average $$\sum_i w_i b_i$$ (I assume $\sum_i w_i=1$).
To see this, do some algebra, as follows. (For convenience we'll let $n$ be the number of terms, thereby sparing ourselves the need to repeatedly write $\sum\limits_i\ 1$.) \begin{align} & {} \qquad \sum_{i=1}^n w_i(x - b_i)^2 = \sum_{i=1}^n w_i(x^2 - 2xb_i + b_i^2) = x^2 - \left(2x\sum_{i=1}^n (w_i b_i)\right) + \left(\sum_{i=1}^n w_i b_i^2\right) \\ \\ \\ & = \underbrace{\left(x^2 -2x\sum_{i=1}^n w_i b_i + \left(\sum_{i=1}^n w_i b_i \right)^2 \right)}_\text{a square} + \sum_{i=1}^n w_i b_i^2 - \left(\sum_{i=1}^n w_i b_i \right)^2\tag{completing the square} \\ \\ \\ & = \underbrace{\left( x - \sum_{i=1}^n w_i b_i \right)^2}_\text{a square} + \sum_{i=1}^n w_i b_i^2 - \left(\sum_{i=1}^n w_i b_i \right)^2. \end{align} The term labeled "a square" is $0$ if $x =\text{the aformentioned weighted average}$, and is otherwise positive. In the succeeding terms, "${}\;x\;{}$" does not appear at all. Therefore the value of $x$ that minimizes the whole expression is the value of $x$ that minimizes the term labaled "a square", and that is the weighted average. the actual value of the minimum is then given by those last two terms.