A conditional distribution related to two normal variables It is given that $\tilde{x} = \theta + \varepsilon$, where $\theta \sim N(y, 1/a)$ and $\varepsilon \sim N(0, 1/b)$. 
Conditional on that we observe a realization $\tilde{x} = x$, what will be the conditional distribution of $\theta$? 
It seems the correct answer would be a normal distribution with a mean of $\frac{ay + bx}{a+b}$ and a variance of $\frac{1}{a+b}$. But I am not sure how to get there. 
I thought that conditional on $\tilde{x} = x$, one can treat $x$ as a constant so that $\theta|_{\tilde{x}=x} \sim N(x,1/b)$. But apparently I got something wrong. 
Can anyone help? Thanks a lot! 
 A: I'm going to assume you intended $\theta$ and $\varepsilon$ to be independent. Omission of that information is a frequent mistake.
Your proposal that the conditional expected value of $\theta$ given $\tilde x = x$ is $x$ would be right if $\tilde x$ and $\varepsilon$ were independent. But if $\theta$ and and $\varepsilon$ are independent, then the correlation between $\tilde x = \theta+\varepsilon$ and $\theta$ is positive, not zero.
Consider the case where the variance of $\theta$ is small and the variance of $\varepsilon$ is large: in that case, $\tilde x$ being a long way from $y$ would more likely result from variation of $\tilde x$ than from $\theta$ being far from $y$, so the conditional expected value of $\theta$ would certainly not be $\tilde x$.
One way to look at conditional distributions in this situation is via Bayes's theorem, which says the posterior distribution is proportional to the product of the prior distribution and the likelihood function. The prior distribution of $\theta$ is proportional to
$$
\sqrt a \exp\left( \frac{-1}2 a^2 (t-y)^2 \right) \,dt.
$$
The likelihood function given the observation that $\tilde x = x$ is
$$
L(t) = \sqrt b \exp\left( \frac{-1}2 b^2 (x - t)^2 \right).
$$
Notice that in the first line above you see $dt$, indicating that this is a measure on the $t$-axis, and no $dt$ appears in the second line; this is not a measure but a pointwise defined function. Multiplying a measure by a pointwise defined function yields a measure, so you expect to see $dt$ again.  
To multiply these, we add the exponents:
\begin{align}
& a(t-y)^2 + b(x-t)^2 \\[10pt]
= {} & (a+b) t^2 - 2(ya+xb)t + \text{constant} \\
& \quad \text{(and in this case “constant" means not depending on $t$)} \\[10pt]
= {} & (a+b) \left( t^2 - 2 \frac{ya+xb}{a+b}  t \right) + \text{a different constant} \\[10pt]
= {} & (a+b) \left( t - \frac{ya+xb}{a+b} \right)^2 + \text{yet another constant}.
\end{align}
We get the posterior distribution
$$
\text{constant} \times \exp\left( (a+b) \left( t - \frac{ya+xb}{a+b} \right)^2 \right) \, dt.
$$
Hence the posterior distribution is a normal distribution with variance $1/(a+b)$ and expected value is $(ya+xb)/(a+b).$
As a way to check this, find
$$
\operatorname{cov}\left( \tilde x, \theta - \frac{ay+b\tilde x}{a+b} \right),
$$
You should get $0$, so they're uncorrelated, and in a multivariate normal setting that means they're independent. If two random variables are independent, then the conditional expected value of the first one given the second, is the first one, so we have
$$
\operatorname{E}\left( \theta - \frac{ay+b\tilde x}{a+b} \mid \tilde x \right) = 0,
$$
and that implies
$$
\operatorname{E}\left( \theta \mid \tilde x \right) - \operatorname{E}\left( \frac{ay+b\tilde x}{a+b} \mid \tilde x \right) = 0.
$$
Since the second conditional expected value is determined by $\tilde x$, we have
$$
\operatorname{E}( \theta \mid \tilde x) -  \frac{ay+b\tilde x}{a+b} = 0.
$$
A: The joint distribution is 
$$
f(\theta,\epsilon) = K(a,b) e^{-\frac12 \left( (y-\theta)^2a + \epsilon^2 b \right)}
$$
where $K(a,b)$ is some normalization factor like $\frac{\sqrt[4]{ab}}{2\pi}$ that we need not care about.
Using $\theta = \tilde{x}   - \epsilon$ we can move over to use $\tilde{x}$ instead of $\theta$ as one variate:$$
f(\tilde{x},\epsilon) = K(a,b) e^{-\frac12 \left( (y-\tilde{x}+\epsilon)^2a + \epsilon^2 b \right)}$$Then we find the marginal distribution of $\epsilon$ by setting $\tilde{x} = x$:
$$f(\epsilon\mid\tilde{x}=x) = K(a,b) e^{-\frac12 \left( (y-x+\epsilon)^2a + \epsilon^2 b \right)}$$
Anticipating that this will be a normal distribution, we can work with just the quantity in the exponent, and complete the square:
$$
-\frac12 \left( (y-x+\epsilon)^2a + \epsilon^2 b \right) =
-\frac12 \left( \epsilon^2(a+b) -2\epsilon(a^2x-ay) +a(y-x)^2 \right)\\
= -\frac12 (a+b) \left( \epsilon^2-2\epsilon\frac{ax-ay}{a+b} + \frac{a(y-x)^2}{a+b} \right)\\
= -\frac12 (a+b) \left( \epsilon-\frac{ax-ay}{a+b}\right)^2 - M(a,b,x,y) 
$$
where $e^{-\frac12 (a+b) M(a,b,x,y)}$ is again a normalization factor that we need not be concerned about (although the careful calculator will keep track to make sure the integral of the marginal distribution indeed works out to one).
We now recognize that 
$$
f(\epsilon) \propto e^{-\frac12 (a+b) \left( \epsilon-\frac{ax-ay}{a+b}\right)^2}
$$
is a normal distribution 
$$N\left( \frac{ax-ay}{a+b},\frac1{\sqrt{a+b}} \right)$$ 
This is different than the answer you said "seems correct." You can tell that this answer is wright because when $a$ and $b$ are very large, the marginal distribution of
$\epsilon$ must be strongly  peaked about $x-y$.
