Variance with correlated variables A simple question that I don't manage to solve:
I can use different methods to measure a magnitude $x$. The results of these methods are correlated and have some uncertainties. Combining the results (and the uncertainties) I obtain $x$ with its uncertainty $\sigma_{x}$. Is it possible that including more methods would increase $\sigma_{x}$?
 A: OK, lets see. It should be enough to consider the case with two measurements, $X$ and $Y$, both unbiased measurements for the quantity $\mu$, so $\DeclareMathOperator{\E}{E} \E X = \E Y = \mu$.  Assume they have variances $\sigma^2_X, \sigma^2_Y$ and covariance
$\sigma_{XY} =\rho \sigma_X \sigma_Y$ where $\rho$  is the correlation between them.
Then use some weighted average to estimate $\mu$:
$$
\hat{\mu} = w_1 X + w_2 Y
$$
where $w_1 + w_2 =1 $ to get an unbiased estimator. Now calculate the variance of $\hat{\mu}$:
$$   \DeclareMathOperator{\var}{Var}  \DeclareMathOperator{\cov}{Cov}
\begin{align}
  \var(\hat{\mu}) &= w_1^2 \var X + w_2^2 \var Y + 2 w_1 w_2 \cov(X,Y)  \\ &=
    w_1^2 \sigma_X^2 + w_2^2 \sigma_Y^2 + 2 w_1 w_2\rho \sigma_X \sigma_Y  \\
   &= (w_1 \sigma_1 + w_2 \sigma_2)^2 + (\rho -1) 2 w_1 w_2 \sigma_X \sigma_Y
\end{align}
$$
This is increasing in $\rho$, so the worst case is if $\rho=1$.
Then we have some cases: first, if $\sigma_X=\sigma_Y$. Then the above expression (with $\rho=1$) becomes simply $\sigma_X^2$, so the extra observation does not help, neither does is harm. But if $\sigma_Y > \sigma_X$, it will harm:  Then we get 
, with $\sigma_Y = \alpha \sigma_X$ and $\alpha > 1$: Above expression becomes
$\sigma_X^2 (w_1 + \alpha w_2)^2 > \sigma_X^2$. You can also make examples of this with $\rho$ close  to 1 and $\sigma_Y > \sigma_X$. On the other hand, when $\rho < 0 $ it can never happen.
