Reversing conditional distribution I want to find $p(x \vert y)$ given $p(y \vert x)$. I am aware of Bayesian formula but I can not understand why the following logic is wrong.
Let $x$ some random variable with pdf $p(x)$ and $\varepsilon \sim N(0,1)$.
Define $y(x) \equiv x + \varepsilon$, then $(y\vert x) \sim N(x,1)$.
But it also should be true that $x=y-\varepsilon$ and because standard normal distribution is symmetric around zero, then it should follow that $(x \vert y) \sim N(y,1)$. Thus we didn't use the information about unconditional distributions, needed for Bayesian formula.
Please, point me where I am wrong. 
Update 1: by Bayesian formula I mean $p(x \vert y) = \frac{p(y \vert x)p(x)}{\int_{-\infty}^{+\infty}p(y \vert x)p(x)dx}$.
Update 2: The following analytical solution should prove that two distributions are indeed not equal.
Let $x \sim N(\mu,\sigma)$, then $p(x)=\frac{e^{-\frac{(x-\mu )^2}{2 \sigma ^2}}}{\sqrt{2 \pi } \sigma }$.
Let $y(x)=x+\varepsilon$ and $\varepsilon \sim N(0,1)$, then $p(y\vert x)=\frac{e^{-\frac{1}{2} (y-x)^2}}{\sqrt{2 \pi }}$.
Hence, $\int_{-\infty}^{+\infty}p(y \vert x)p(x)dx=\frac{e^{-\frac{(y-\mu )^2}{2 \left(\sigma ^2+1\right)}}}{\sqrt{2 \pi } \sqrt{\frac{1}{\sigma ^2}+1} \sigma }$ and therefore by Bayesian formula $p(x\vert y)=\frac{\sqrt{\frac{1}{\sigma ^2}+1} \exp \left(-\frac{(x-\mu )^2}{2 \sigma ^2}-\frac{1}{2} (y-x)^2+\frac{(y-\mu )^2}{2 \left(\sigma ^2+1\right)}\right)}{\sqrt{2 \pi }}$. Finally, this is not equal to pdf of $N(y,1)$, which is $\frac{e^{-\frac{1}{2} (x-y)^2}}{\sqrt{2 \pi }}$. 
 A: The best is to distinguish between random variables and values taken by them. So let me reformulate the question:
We have three random variables, $X$ and $$Y=X+\varepsilon,$$ and $\varepsilon$ is of $N(0,1)$. If we may, we assume that $X$ and $\varepsilon$ are independent. The conditional distribution of $Y$ given that $X=x$ is then, indeed $N(x,1)$.
On the other hand $$X=Y-\varepsilon$$
and let's calculate $X$'s conditional distribution given that $Y=y$.
$$F_{X\mid Y=y}(x)=P(X<x\mid X+\varepsilon=y)=P(X<x\mid X=y-\varepsilon)=$$
$$=P(y-\varepsilon<x\mid X=y-\varepsilon)=P(y-\varepsilon<x)$$
because of the independence of $X$ and $\varepsilon$. From here,
$$F_{X\mid Y=y}(x)=P(y-\varepsilon<x)=P(\varepsilon>y-x)=P(\varepsilon<x-y)$$
because of symmetry. And this is the distribution function of $N(y,1)$. So, there is no mistake. 
The only thing is that independence had to be assumed. Also, the symmetric nature of $N(0,\sigma)$ was exploited.

EDIT
On the necessity of independence.
Let $\epsilon^-$ and $\epsilon^+$ be two random variables of "half normal $\pm$" distribution. The distributions are defined as follow
$$F_{\epsilon^+}(x)=\begin{cases}2N(0,1)(x)&\text{ if } x\geq 0\\0&\text{otherwise}\end{cases}.$$
($\epsilon^{-}$'s pdf is similar but it is $0$ for $x$s less than zero...) Also, assume that $\epsilon^{\pm}$ are independent of $X$. Now, assume that there is an $x_0>0$ so that $P(X<x_0)=P(X\geq x_0)=\frac 12$ and define $$\varepsilon=\begin{cases}\varepsilon^-&\text{ if }&X<x_0\\\epsilon^+&\text{ if }&X\geq x_0.\end{cases}.$$
The distribution of $\varepsilon$ is $N(0,1)$ but it is not independent from $X$.
Let't calculate $Y$'s conditional distribution given that $X=x$:
$$P(X+\epsilon<y\mid X=x)=P(\varepsilon<y-x\mid X=x)=\begin{cases}F_{\epsilon^-}(y-x)&\text{ if }& x<x_0\\ F_{\epsilon^+}(y-x)&\text{ if }& x\geq x_0.\end{cases}$$
For a given $x>x_0$ the conditional pdf of $Y$ looks like this
A: The problem comes from the second part of the argument where you write $x=y-\varepsilon$. Because $y$ and $\varepsilon$ are not independent variables, $P_{X|Y}(x|y)$ is not just a shift of $P_\varepsilon$.
To convince yourself that is the source of the problem, think of the following argument:
If $y = x+\varepsilon$, and $x$ and $\varepsilon$ are independent variables, then $y$ will have a distribution with a variance larger than that of $x$.
If $x = y - \varepsilon$, and $y$ and $\varepsilon$ are independent variables, then $x$ will have a distribution with the larger variance.
A: Take $f(x,y)=p(y|x)$ as given conditional PDF and $p_X{x}$ as PDF of $X$. therefore:
$$p(x,y)=p(y|x)p_X(x)\to p_Y(y)=\Sigma_x p(y|x)p_X(x)\to p(x|y)={{p(y|x)p_X(x)}\over{\Sigma_x p(y|x)p_X(x)}}$$
So this question requires extra information. Now using your method the question is:
"What if $\epsilon\nsim N(\mu,\sigma^2)$?"
A: Let's clean up notation first; I will use $Z$ instead of $\varepsilon$. We have 2 independent random variables $X$ and $Z$. To simplify the following I'll assume that $X$ and $Z$ are discrete. Now we define $Y=X+Z$
The conditional PMF of $Y$ given $X$ is 
$$p_{Y|X}(y|x)=P(Y=y|X=x)=P(X+Z=y|X=x)=P(x+Z=y)=P(Z=y-x)=p_Z(y-x)$$
so the conditional PMF $p_{Y|X}(y|x)$ if the same as $p_Z(y)$ shifted right by $x$.
The conditional PMF of $X$ given $Y$ is 
$$p_{X|Y}(x|y)=P(X=x|Y=y)=P(Y-Z=x|Y=y)$$
We can't proceed from here to
$$P(Y-Z=x|Y=y) \overset{?}{=} P(y-Z=x)=P(Z=y-x)=p_Z(y-x)$$
because $Y=X+Z$ and $Z$ are dependent.
