1
$\begingroup$

Defining the barycentre and finding its variance

I have a set of $N$ points at the locations $x_i$ which has weights $W_i$, $i=1,\ldots, N$ and want to find the barycenter (or center of gravity)

$$ B = {\sum_{i=1}^N W_i x_i\over \sum_{i=1}^N W_i}.$$

I want to find the variance of $B$. I assume that $W_i$ are i.i.d. and positive with expected value $Ew_i=\mu$ and variance $Var[W_i]=E[w_i-\mu]^2=\sigma^2 $. I also assume that $N$ is large, so $\sum_{i=1}^N w_i$ is well approximated by $N\mu$ (justified below).

When I do the calculations (see one example below) I end up with $$ Var [B] = {\sigma^2 \over N^2 \mu^2} \sum_{i=1}^N x_i^2. $$

The contradiction

As explained below, I think that the barycentre variance should be independent of a shift of all the $x_i$, that is $Var[B]$ should not change if all $x_i$ is shifted to $x_i+c$. But $\sum_{i=1}^N x_i^2$, and therefore my estimated $Var[B]$, will change a shift of the $x_i$.

Where is the error: In my calculations, or the expectation that the variance should be independent of shift in $x$.


One derivation of the variance

$$Var [B ] = Var \left[{\sum_{i=1}^N W_i x_i \over \sum_{i=1}^N w_i}\right] = {1\over N^2 \mu^2} \sum_{i=1}^N x_i^2 Var [W_i] ={\sigma^2\over N^2 \mu^2} \sum_{i=1}^N x_i^2$$

Here I used that $Var [\sum_i X_i] = \sum_i Var[ X_i]$ when the $X_i$ are independent, that $Var [cX]=c^2 Var[X]$, and that $Var[ w_i] =\sigma^2$.

I have also done the same in a more direct fashion, with the same result. It is kind of messy, so I have omitted it.

Why do I expect $Var [B]$ to be invariant to a shift of $x_i$?

The expected value of $B$ is the mean of the $x_i$:

$$E[B]=E\left[{\sum_{i=1}^N E[W_i] x_i\over \sum_{i=1}^N W_i}\right] ={\sum_{i=1}^N E[W_i] x_i\over N\mu} ={\mu\sum_{i=1}^N x_i\over N\mu}={\sum_{i=1}^N x_i\over N}=\bar x_i.$$

Moving all $x_i$ by $c$ will move every barycentre estimate by $c$: $$ \tilde B = {\sum_{i=1}^N W_i (x_i+c)\over \sum_{i=1}^N W_i} = {\sum_{i=1}^N W_i x_i\over \sum_{i=1}^N W_i} +{\sum_{i=1}^N W_i c\over \sum_{i=1}^N W_i}= {\sum_{i=1}^N W_i x_i\over \sum_{i=1}^N W_i} +c=B+c.$$

The variance of $B$ is $Var[B]=E[(B-E[B])^2]$. Moving all the $x_i$ by $c$ adds $c$ to $B$ and $E[B]$, so $B-E[B]$ and the variance should stay the same.

To check this I did a MATLAB simulation. In that simulation the estimated variances and barycentres (after i remove the displacement $c$) are identical.

Why do I think $\sum_{i=1}^N w_i =N\mu$ is a good approximation

Although the variance of $\sum_{i=1}^N w_i$ grow with $N$, this means that the standard deviation grow with $\sqrt N$. And since the expected value grow with $N$, the relative spread will decrease.

It is the relative spread that is important for this case. Say that for one $N$ the expected value is 100 and the standard deviation is 1. Then, if $\sum_{i=1}^N w_i $ miss $N\mu$ with one standard deviation my estimate of $B$ is off by 1%. If I increase N by a factor of a 100, the expected value will be 10 000 and the standard deviation will be 10. So if $\sum_{i=1}^N w_i $ miss $N\mu$ with one standard deviation my estimate of $B$ is off by 0.1%.

Which mean that for large enough N the error will be negligible.


Context

I'm trying to repeat a derivation for the variance of an estimated arrival time from this paper. The barycentre is used to find the time when the echo from the seafloor is received for seabed mapping echo sounder.

As a simplified model, the part of the time sequence used in the barycentre calculation can be modelled as a sequence of independent, identically (Rayleigh) distributed, amplitude values. These amplitude values are equally spaced in time.

In this question the $W_i$ represent the amplitudes, the $x_i$ it the sample times and $B$ is the estimated time of arrival. The paper states that it can be show analytically that

$$Var[B]= \frac{\Delta x^2}{12(N+1)}\left(\frac{4}{\pi}-1\right)N(N+2),$$

where $\Delta x$ is the distance between the $x_i$ ($x_i=x_0+i \Delta x$). I have translated the equation to the notation used in this question. No reference is given, although I am searching in related papers.

The middle factor in that equation $\left(4/\pi-1\right)$ is equal to the ratio the variance and the mean value squared of a Rayleigh distributed variable.

$\endgroup$

2 Answers 2

1
$\begingroup$

There are a couple of mistakes in your work here.

In your derivation you replace the denominator, $\sum W_i$ with its expectation value. You say you do this because it is a good approximation, but you are ignoring the variance it contributes. Would you still do that if the numerator was a constant? If so, you would have found that $V[\hat{X}]=0$.

The more important issue is that you are assuming the $W_i$ are independent. If we make the replacement, $$F_i = {W_i\over\sum W_i}$$ then, $$B = \sum x_i F_i$$ Because $\sum F_i = 1$, there are only $n-1$ independent random variables $F_i,\ \ i =2...n$. The other random variable is $$F_1 = 1 - \sum_{i=2}^{n}F_i$$ Write $B$ in terms of the remaining independent random variables: $$B = x_1\left(1 - \sum_{i=2}^{n}F_i\right)+\sum_{i=2}^n x_i F_i=x_1+\sum_{i=2}^n (x_i-x_1) F_i$$ Then, $$E[B] = x_1 +E[F]\sum_{i=2}^n (x_i-x_1)= x_1 + {1\over n}\sum_{i=2}^n (x_i-x_1) = {1\over n}\sum_{i=1}^n x_i = \bar{x}$$ and $$V[B] = V[F]\sum_{i=2}^n (x_i-x_1)^2$$

For this example the variance is invariant under translation of $x_i$. In reducing the degrees of freedom to $n-1$, the term $x_1$ is treated differently from the rest; note that $V[F_1]=(n-1)V[F]$.

$\endgroup$
12
  • $\begingroup$ Thanks for your suggestions! I updated the notation to make my question easier to understand, although not following you completely. I disagree with your two other issues, so I added justification for my claim about the simplification in the denominator and why I expect the variance to change when $x_i\rightarrow x_i+c$. I think my simplification is valid in the denominator, although very bad in the numerator as you point out. I also think the translation $f_i \rightarrow f_i+c$ is impossible since the way $F_i$ is defined $\sum F_i$ must be 1. $\endgroup$
    – Mixopteryx
    Commented Nov 9, 2017 at 21:17
  • $\begingroup$ Okay I removed the incorrect side remark about translating $f_i$. It doesn't change the argument. If you reformulate your problem so that the denominator is a constant, it still does not change the argument. The variance for the quantity you now call $B$ comes from the variance of the random variables $F$. Please look at the equation for $\hat{F}$. The variance clearly increases if all $x_i$ are increased. Are you surprised that the variance of $\hat{F}$ is affected when you shift the $x_i$? That equation is equivalent to your equation for B - so the same applies for $B$. $\endgroup$
    – Dean
    Commented Nov 9, 2017 at 22:30
  • $\begingroup$ I'm sorry, but I still don't follow. Isn't it such that since $F_i=W_i/\sum W_i$, we have that $\sum F_i=\sum W_i/W_i =1$? Therefore $c\sum F_i=c$ and the last equation in your edited answer is an equality and my argument hold. To check my intuition I did a simulation and the variance were unchanged when I shifted the $x_i$. I added the link to my code in my answer. $\endgroup$
    – Mixopteryx
    Commented Nov 10, 2017 at 7:28
  • 1
    $\begingroup$ Yes, my answer incorrectly assumed that the $F_i$ were independent. I have edited the answer above to use the $B$ symbol and to treat the $F_i$ correctly. Hopefully now it is clear where the error was made in your calculation... $\endgroup$
    – Dean
    Commented Nov 10, 2017 at 19:42
  • $\begingroup$ Brilliant! This is a huge step forward for my understanding, and it seems correct. I expect that I will accept it as soon as I find the time to work trough it properly. Then I can move on to the problem of finding $V[F]$ now that I have learned that I cannot ignore the variance contribution of the denominator... $\endgroup$
    – Mixopteryx
    Commented Nov 10, 2017 at 22:00
0
$\begingroup$

Through this PhD thesis (chapter 3.2.3) i found the following solution.

For simplicity, define $$F_i = {W_i\over\sum W_i}, \text{ so that } B=\sum_{i=1}^N F_i x_i.$$ (As Deans answer suggest)

Then $$E[(B-E[B])^2]={\sum_{i=1}^N \sum_{j=1}^N (x_i-\bar x) (x_j-\bar x) E[(F_i-E[F_i]) (F_j-E[F_j])} $$ $${=\sum_{i=1}^N (x_i-\bar x)^2 E[(F_i-E[F_i])^2]} {=\sum_{i=1}^N (x_i-\bar x)^2 Var[F_i]}$$ if we assume that the $F_i$ are uncorrelated. $\bar x$ is the mean of the $x_i$.

The first step holds since $${\sum_{i=1}^N \sum_{j=1}^N (x_i-\bar x) (x_j-\bar x) E[(F_i-E[F_i]) (F_j-E[F_j])}=E\left[\left(\sum_{i=1}^N (x_i-\bar x) (F_i-E[F_i])\right)^2\right],$$ and $$\sum_{i=1}^N (x_i-\bar x) (F_i-E[F_i])=\sum_{i=1}^N( x_iF_i-x_iE[F_i]) + \bar x \sum_{i=1}^N(E[F_i]-F_i)=B-E[B].$$ Where I used that $\sum_{i=1}^NE[F_i]=1$ and $\sum_{i=1}^NF_i=1$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .