Inconsistency in two-sided hypothesis testing Suppose you have two sets of data with known population variances and want to test the null hypothesis that two means are equal, ie. $H_{0}: \mu_{1} = \mu_{2}$ against $H_{1}: \mu_{1} > \mu_{2}$. There's a certain way I want to think about it, which is the following:
\begin{align}
P(\mu_{1} > \mu_{2}) &= P(-(\mu_{1} - \mu_{2}) < 0) \\
&= P\left(\frac{\bar{x}_{1} - \bar{x}_{2} - (\mu_{1} - \mu_{2})}{\sigma_{\delta \bar{x}}}<\frac{\bar{x}_{1} - \bar{x}_{2}}{\sigma_{\delta \bar{x}}} \right) \\
&=P\left(z < \frac{\bar{x}_{1} - \bar{x}_{2}}{\sigma_{\delta \bar{x}}}  \right)
\end{align}
To me, this 'derivation' makes it perfectly clear what's actually going on. You're actually calculating the probability that $H_{1}$ is true and not just blindly looking up some $z$-score. However, now suppose that $H_{1}: \mu_{1} \neq \mu_{2}$. The problem with this is that the method I just described doesn't seem to work. If I write
$$
P(\mu_{1} \neq \mu_{2}) =  P(\mu_{1} < \mu_{2}) + P(\mu_{1} > \mu_{2})
$$
Then all that happens is $P(\mu_{1} \neq \mu_{2}) = 1$. I think I'm probably not interpreting the above equation correctly.
 A: In your way of thinking, you decided to set
$$z = \frac{\bar{x}_1 - \bar{x}_2 - (\mu_1 - \mu_2)}{\sigma_{\delta \bar{x}}},$$
and from this you conclude (correctly, I think) that 
$P(\mu_1 > \mu_2) = P\left(z < \frac{\bar{x}_1 - \bar{x}_2}{\sigma_{\delta \bar{x}}}\right).$
The question is, if you cannot assign a probability distribution to $\mu_1 - \mu_2,$ how do you compute $P\left(z < \frac{\bar{x}_1 - \bar{x}_2}{\sigma_{\delta \bar{x}}}\right)$?
And of course if you can assign a probability distribution to $\mu_1 - \mu_2,$
then you can use that distribution to compute $P(\mu_1 - \mu_2 > 0)$ directly.
Note that if $P(\mu_1 = \mu_2) > 0,$ then
$$P\left(z = \frac{\bar{x}_1 - \bar{x}_2}{\sigma_{\delta \bar{x}}}\right) > 0,$$
because you have defined $z$ in such a way that it is simply 
$\frac{\bar{x}_1 - \bar{x}_2}{\sigma_{\delta \bar{x}}}$ 
if the two populations have the same mean.
When you conclude that $P(\mu_1 \neq \mu_2) = P(\mu_1 < \mu_2) + P(\mu_1 > \mu_2) = 1,$
you are assuming that $z$ has zero probability to equal 
$\frac{\bar{x}_1 - \bar{x}_2}{\sigma_{\delta \bar{x}}}$ exactly.
This would be true if $z - \frac{\bar{x}_1 - \bar{x}_2}{\sigma_{\delta \bar{x}}}$ 
had a continuous distribution, but how do we know it does?
Edit: There are some quite reasonable motivations (from a practical point of view,
if not a mathematical one) for attempting some kind of approach to coming up with
a value for $P(\mu_1 > \mu_2)$ as explained in the question. The main motivation is
that this is the way we seem to want to be able to think about statistics:
just how much weight (i.e. likelihood) should I assign to the possibility that
certain facts are true? Unfortunately it's often very difficult to make a convincing
case for a particular value of such a likelihood.
 Instead, what frequentist statistics gives us is an
apparently roundabout statement that if a certain fact were not true
(that is, if that fact's "null hypothesis" were true instead), it would
have been extremely unlikely for us to have made the observations we just made.
A more precise and succinct explanation is given in this answer to another question.
To test the hypothesis that $\mu_1 > \mu_2$ in the posted question, 
we can define the null hypothesis as $\mu_1 \leq \mu_2.$
Now, having obtained samples from the two populations, how likely is it that we
would have gotten samples "like those" if the null hypothesis were true?
If $\bar x_1 \leq \bar x_2$ the answer is "likely enough," so we only have an interesting
statistical test in the case where $\bar x_1 > \bar x_2.$
Assuming that's the kind of sample results we got, then
among all possible ways the null hypothesis could be true, the one that gives us the
best chance to obtain samples "like" the ones we did is if $\mu_1 = \mu_2.$
But if we assume that $\mu_1 = \mu_2,$ then prior to taking the samples,
$\frac{\bar{x}_1 - \bar{x}_2}{\sigma_{\delta \bar{x}}}$
was a random variable with a standard normal distribution (mean zero, variance $1$).
Suppose that having taken our samples, we find that
$\frac{\bar{x}_1 - \bar{x}_2}{\sigma_{\delta \bar{x}}} = 2.38.$
That's a relatively extreme value for a standard normal variable; $99$ times out of $100$
the value of a standard normal variable will be less than that.
In fact, the probability is $0.99134$ that a standard normal variable will
have a value less than $2.38.$ (I know this because someone computed that probability
and put it in a table, and I looked it up there.)
There is therefore less than a $1$% chance that we would have observed a
sample mean $\bar x_1$ so much larger than $\bar x_2$ if the population mean
$\mu_1$ were not actually at least a little bit larger than $\mu_2.$
We therefore reject the null hypothesis.
Using samples drawn from populations with continuous distributions, it appears impossible to test statistically whether the means of two populations are exactly the same, because even if they were the same, with probability $1$ we would still observe different sample means. (See also this answer on that topic.)
There is another kind of statistics called Bayesian statistics that (as far as I understand it) does assign probabilities to the truth of statements that you might want to prove or disprove, but only by using observations of experiments to modify probability assignments that one was able to make before the experiment.
A: If is probably that $\mu_1<\mu_2$, then  $\mu_1\geq\mu_2$ is rejected, then not is possible that holds $\mu_1<\mu_2$ and $\mu_1>\mu_2$ simultaneusly.
A: You have
$$
\frac{\bar X_1-\bar X_2}{\sigma_{\delta \bar X}}\sim N(0,1). 
$$
If you want to use $\bar X_1-\bar X_2$ as the test statistic, then you would reject the null hypothesis if $\bar X_1-\bar X_2>c$, for some critical value $c$.  Often one chooses $c$ by deciding how much probability of Type I error one is willing to tolerate --- a subjective economic decision.  One can then think about probability of Type II error as a function of $\mu_1-\mu_2$, and that probability would depend on the value of $c$ and on the two sample sizes.
None of this gives you a probability distribution of $\mu_1-\mu_2$.  For that you'd need a prior distribution, and then you'd have to find the posterior distribution, i.e. the conditional distribution given the two samples.
PS: A bit of back-of-the-envelope scratchwork suggests that the likelihood-ratio test statistic may be:
$$
\left.\begin{cases} \left(\dfrac{\bar X_1 - \bar X}{\sigma_2/\sqrt{n_1}}\right)^2 + \left(\dfrac{\bar X_2-\bar X}{\sigma_2/\sqrt{n_2}}\right)^2 & \text{if }\bar X_1\ge \bar X_2, \\[10pt]
{}\qquad 0 & \text{otherwise},
\end{cases}\right\}
$$
where $\bar X$ is simply the grand mean $\dfrac{n_1 \bar X_1 + n_2 \bar X_2}{n_1+n_2}$.
