a simple markov chain model for treatment Suppose doctors are experimenting with two treatments without knowing which one is better.
Call treatments $A$ and $B$. One is likely better, but we don’t know which one. A series of patients will each be given one of the treatments. We aim to find a strategy that ensures that as many as possible of the patients are given the better treatment — though we don’t know which one this is.
Given that for any patient, $\mathbb P( \text{ treatment A is success})=\alpha$ and
$\mathbb P( \text{ treatment B is success})=\beta$, $0<\alpha, \beta<1$.
For any patient, doctors can choose with equal probability which treatment to go ahead with. Let $P_R$ be the overall probability of success for each patient. so $P_R=\frac{1}{2}(\alpha+\beta)$.
There is another strategy where for the first patient doctor chooses treatment $A$ and $B$ with equal probability at random; if the $n^{th}$ patient is given treatment $A$ and it is successful, they use the treatment $A$ for the $(n+1)^{th}$ patient too.
A similar rule is patient $n$ is given treatment $B$, and if it is successful, they use the treatment $B$ for the $(n+1)^{th}$ patient too. If not, switch to treatment $A$ for the    $(n+1)^{th}$ patient.
So, the treatment system can be expressed with a Markov chain with $4$ states, right?
$\{A_s, A_f, B_s, B_f \}$
Could anyone tell me how if I have written the correct transition probabilities?
$\mathbb P( X_{n+1}=A_s \mid X_n=A_s)=\frac{1}{2}\alpha$
$\mathbb P( X_{n+1}=A_s \mid X_n=B_f)=\frac{1}{4}\alpha(1-\beta)$
$\mathbb P( X_{n+1}=A_s \mid X_n=B_s)=0$
$\mathbb P( X_{n+1}=B_s \mid X_n=B_s)=\frac{1}{2}\beta$
$\mathbb P( X_{n+1}=B_s \mid X_n=A_f)=\frac{1}{4}(1-\alpha)\beta$
$\mathbb P( X_{n+1}=B_s \mid X_n=A_s)=0$
Where $A_s$ means that the patient was given treatment $A$ and was successful. $f$ indicates failure.
Define $P_T$ to be the long-run probability of success in this strategy.
Could anyone tell me how to find the equilibrium distribution $\pi$ for the process? Find $P_T$, the long-run probability of success for each patient in this strategy.
Thank you for your help.
 A: Inaccuracies in the transitions
I don't think your transition probabilities are correct.
For instance, condition on $X_n = A_s$, $X_{n+1} \in \{A_s, A_f\}$, but you haven't defined a probability for $\mathbf P[X_{n+1} = A_f | X_n = A_s]$.
Further, still conditioning on $X_n = A_s$ the respective probabilities are simply the probability $A$ succeeds/fails, so:
$$\begin{align*}
\mathbf P[X_{n+1} = A_s | X_n = A_s] & = \alpha,\\
\mathbf P[X_{n+1} = A_f | X_n = A_s] & = 1-\alpha.
\end{align*}$$
So we don't need the factor $\frac12$ you have in your first formula.
Complete Transition Matrix
Let $P$ denote the transition matrix, and I'll assume the order of the four states that you provided $\{A_s, A_f,B_s, B_f\}$, then thinking through each scenario, as above, you can show:
$$P = \left(
\begin{matrix}
\alpha & 1-\alpha & 0 & 0 \\
 0 & 0 & \beta & 1-\beta \\
 0 & 0 & \beta & 1-\beta\\
 \alpha & 1-\alpha & 0 & 0
\end{matrix}
 \right)$$
Calculating the stationary distribution
This is an irreducible, aperiodic, Markov chain on a finite state space, so it is guaranteed to have a unique stationary distribution, $\pi$ (e.g. Theorem 4.9 in Levin and Peres).
Further, by definition of a stationary distribution, it satisfies:
$$\pi P = \pi.$$
Applying this to the matrix above, we have the system of equations
$$
\begin{align*}
\alpha(\pi_1 + \pi_4) = \pi_1 \\
(1-\alpha)(\pi_1 + \pi_4) = \pi_2 \\
\beta(\pi_2 + \pi_3) = \pi_3 \\
(1-\beta)(\pi_2 + \pi_3) = \pi_4
\end{align*}
$$
Combining the first two equations, we find that $\pi_2 = \pi_4$, which we'll denote $p$. Then respectively substituting $p$ for $\pi_4$ for $\pi_2$ in the second and fourth equations we find:
$$
\begin{align*}
\pi_1 = \frac{\alpha}{1-\alpha}p \\
\pi_3 = \frac{\beta}{1-\beta}p
\end{align*}
$$
Since $\pi_1 + \pi_2 + \pi_3 + \pi_4 =1$ we have:
$$1 =\frac{\alpha}{1-\alpha}p + p + \frac{\beta}{1-\beta}p + p,$$
Equivalently:
$$p = \frac{1}{2 + \frac{\alpha}{1-\alpha} + \frac{\beta}{1-\beta}}.$$
So we have the stationary distribution:
$$
\pi = 
\frac{1}{2 + \frac{\alpha}{1-\alpha} + \frac{\beta}{1-\beta}}
\left(\begin{matrix}
\frac{\alpha}{1-\alpha} \\
1 \\
\frac{\beta}{1-\beta} \\
1
\end{matrix}\right)
$$
Success Probability
Having run the Markov chain long enough, (approximately) any patient can expect to be in each state according to the stationary distribution $\pi$. Successful patients are those in states $A_s,\, B_s$, which correspond to $\pi_1,\pi_3$.
So the probability of success in this strategy is
$$ \pi_1 + \pi_3 = \frac{\frac{\alpha}{1-\alpha} + \frac{\beta}{1-\beta}}{2 + \frac{\alpha}{1-\alpha} + \frac{\beta}{1-\beta}}=\frac{\alpha+\beta-2\alpha\beta}{2-\alpha-\beta}$$
