How do I combine multiple noisy measurement? Assuming there's a variable I want to measure, but I have only very noisy instrument to do so. So I want to take multiple measurements so that I have a better chance to recover the state of the variable. Hopefully, with each measurement, my instrument can report the result as a Gaussian distribution , with the mean to be the most likely state of variable and the standard deviation suggests a rough possible region of the state. 
My problem now is that I don't know how to combine these multiple measurements to get a sensible answer. My guess is that it would be nice if I can get a new gaussian from these results, with the mean centered at the expectation value of the state of the variable, and a standard deviation to reflect how confident I am about the result...
I tried to teach myself about gaussians, and probabilities, but I just couldn't get my head around...please can someone help me?
 A: Actually, you are measuring something of the form
\begin{equation}
X_i = v + b_i \, ,
\end{equation}
where $v$ is the deterministic value you want to measure, and $b_i$ is the value of a Gaussian noise at the $i$th measurement. If the measurements are independent from each other, then simply take the arithmetic mean
\begin{equation}
\overline{X}_n = \frac{1}{n} \sum_{i=1}^n X_i \, ,
\end{equation}
which has a normal distribution, by linear combination of Gaussian variables.
If your system is ergodic (broadly speaking, the system and the noise do not change behavior over time, i.e. $v$ and the distribution of $b_i \sim\mathcal{N}(\mu,\sigma^2)$ do not change over time), then the expected value and the variance of $\overline{X}_n$ are
\begin{equation}
E(\overline{X}_n) = \frac{1}{n} \sum_{i=1}^n E(X_i) = v+\mu \, ,\\
V(\overline{X}_n) = \frac{1}{n^2} \sum_{i=1}^n V(X_i) = \frac{\sigma^2}{n} \, .
\end{equation}
The random variable $\overline{X}_n$ has a normal distribution $\mathcal{N}(v+\mu, \sigma^2/n)$. If you have a reference measurement where the value $v$ is known, e.g. deduced from another measurement technique, then you can estimate $\mu$ and deduce by how much the noise modifies the mean of $\overline{X}_n$.
If the noise distribution changes at each measurement, $b_i\sim \mathcal{N}(\mu_i,{\sigma_i}^2)$ for each $i$, then the arithmetic mean $\overline{X}_n$ has a normal distribution $\mathcal{N}(v + \overline{\mu}_n, {\overline{\sigma^2}}_n/n)$. Alternatively, one can compute the weighted and centered mean
\begin{equation}
\widetilde{X}_n = \sum_{i=1}^n w_i \left(X_i - \mu_i\right) \quad\text{with the weights}\quad w_i = \frac{{\sigma_i}^{-1}}{\sum_{j=1}^n {\sigma_j}^{-1}} \, ,
\end{equation}
which reduces to the arithmetic mean $\overline{X}_n$ when $\mu_i = 0$ and $\sigma_i = \sigma$ for all $i$. The expected value and the variance of $\widetilde{X}_n$ are
\begin{equation}
E(\widetilde{X}_n) = v \sum_{i=1}^n w_i = v \, ,\\
V(\widetilde{X}_n) = \sum_{i=1}^n {w_i}^2 {\sigma_i}^2 = n \left(\sum_{j=1}^n {\sigma_j}^{-1}\right)^{-2} .
\end{equation}
A: I think the previous answer was clear but I was confused on why $\widetilde{X_n}$ was defined so. 
Let us start afresh on the problem. So there are multiple (focus on two first) measurements and we want to aggregate it to get a better estimate on the true state (with smallest variance). Let us suppose if both measurement were independent and identical with following form. Let us derive the best linear unbiased estimator for this simple case.
$X_i = \nu + \sigma Z_i$ where $Z_i$ is standard normal (mean 0 and variance 1). So $X_i$ has mean $\nu$ and variance $\sigma^2$.
By symmetry, a natural choice for a good linear estimator is equal weight of 0.5.
$$\overline{X} = \frac{X_1+ X_2}{2}$$
This estimator is unbiased as $\mathbb{E}\overline{X} = \nu$. Also the variance is $\frac{\sigma^2}{2}$ This is actually the best linear estimator as it has lowest possible variance. 
Now, when the measurements have difference variances, that is when things get tricky. Let us take a simple case with two measurements as follows.
$$X_1 = \nu + \sigma_1^2 Z_1$$
$$X_2 = \nu + \sigma_2^2 Z_2$$
$Z_1$ and $Z_2$ are standard normal. An unbiased estimator for the parameter $\nu$ is given by
$$\widetilde{X} = w_1X_1+ (1-w_1) X_2$$
The parameter $w_1$ lies in $(0,1)$. The mean as discussed before is $\nu$. The expression for variance is given by
$$Var(\widetilde{X})=w_1^2 \sigma_1^2 + (1-w_1)^2 \sigma_2^2$$
Using first order conditions, the best linear estimator (or optimal value of $w_1$) is  given by
$$w_1 = \frac{\sigma_2^2}{\sigma_1^2+\sigma_2^2}$$
In general, when you have multiple measurements the weights are given by,
$$w_i = \sigma_i^{-2}\left(\sum_j \sigma_j^{-2}\right)^{-1}$$
To get the expression use KKT conditions on the condition that $\sum w_i =1$
Note the previous answer has some other weights. I dont know why.
