Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I think if we want to calculate $\min_x \sum_i (b_i - x)^2$, the answer should be the mean of $b_i$, right? Now if we add a weight to each term and make it $\min_x \sum_i w_i(b_i - x)^2$, what's the closed form solution for $x$ then?

share|improve this question
    
Differentiate, set to zero, solve for $x$. Which part is unfamiliar? –  Gerry Myerson Dec 13 '11 at 0:36
    
@GerryMyerson: Yeap...turns out my brain stuck at that point...seems a trivial problem... –  derekhh Dec 13 '11 at 0:39
    
At which point? The differentiation? the setting equal to zero? the solving for $x$? –  Gerry Myerson Dec 13 '11 at 2:06
    
@GerryMyerson : There's no denying that what you propose will solve the problem. I took a course in graduate school where we did a gillion exercises that included generalizations of this involving matrices, in which the (generalization of) the method you suggest would have been a nightmare. Various things involving designs of experiments in which there were lots of Wishart matrices. (All this reminds me that there's a lot of stuff not covered in Wikipedia. There's still no "expected mean squares article.) So I am partial to completing the square. –  Michael Hardy Dec 13 '11 at 2:10

1 Answer 1

up vote 2 down vote accepted

It's just the weighted average $$ \sum_i w_i b_i $$ (I assume $\sum_i w_i=1$).

To see this, do some algebra, as follows. (For convenience we'll let $n$ be the number of terms, thereby sparing ourselves the need to repeatedly write $\sum\limits_i\ 1$.) $$ \begin{align} & {} \qquad \sum_{i=1}^n w_i(x - b_i)^2 = \sum_{i=1}^n w_i(x^2 - 2xb_i + b_i^2) = x^2 - \left(2x\sum_{i=1}^n (w_i b_i)\right) + \left(\sum_{i=1}^n w_i b_i^2\right) \\ \\ \\ & = \underbrace{\left(x^2 -2x\sum_{i=1}^n w_i b_i + \left(\sum_{i=1}^n w_i b_i \right)^2 \right)}_\text{a square} + \sum_{i=1}^n w_i b_i^2 - \left(\sum_{i=1}^n w_i b_i \right)^2\tag{completing the square} \\ \\ \\ & = \underbrace{\left( x - \sum_{i=1}^n w_i b_i \right)^2}_\text{a square} + \sum_{i=1}^n w_i b_i^2 - \left(\sum_{i=1}^n w_i b_i \right)^2. \end{align} $$ The term labeled "a square" is $0$ if $x =\text{the aformentioned weighted average}$, and is otherwise positive. In the succeeding terms, "${}\;x\;{}$" does not appear at all. Therefore the value of $x$ that minimizes the whole expression is the value of $x$ that minimizes the term labaled "a square", and that is the weighted average. the actual value of the minimum is then given by those last two terms.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.