Finding the transition probability matrix, two switches either on or off.. Each of two switches is either on or off during a day. On day n, each switch will independently be on with probability
[1+number of on switches during day n-1]/4
For instance, if both switches are on during day n-1, then each will independently be on with probability ¾. What fraction of days are both switches on? What fraction are both off?
I am having trouble finding the transition probabilities. I know what they all are (I have looked at the solution), but I don't understand how you get the values for each P_ij... I can easily find the stationary probabilities after finding the transition probability matrix.. Can anyone help me guide through the transition steps?
 A: The state space is $(x_n^1,x_n^2)$ which are the states of bulbs 1 and 2 being on/off at time $n$. The transition probability matrix is found as follows, where the ordering of the states is $(0,0), (0,1), (1,0)$ and $(1,1)$.
$$P=\left[\begin{array}\
\frac{9}{16} & \frac{3}{16} & \frac{3}{16} &\frac{1}{16}\\
\frac{1}{4} & \frac{1}{4} & \frac{1}{4} & \frac{1}{4}\\
\frac{1}{4} & \frac{1}{4} & \frac{1}{4} & \frac{1}{4}\\
\frac{1}{16} & \frac{3}{16} & \frac{3}{16} &\frac{9}{16}\\
\end{array}\right]$$
To find the fraction of time both bulbs are off/on, you need to solve $\pi P=\pi$ and find $\pi(0,0)$ and $\pi(1,1)$. The stationary probabilities turn out to be $\pi(0,0)=\pi(1,1)= \frac{2}{7}$ and $\pi(0,1)=\pi(1,0)=  \frac{3}{14}$.
A: If the probability for a switch to be on is $p$, the probabilities for $0$, $1$ or $2$ switches to be on are $(1-p)^2$, $2p(1-p)$ and $p^2$, respectively. If $0$, $1$ or $2$ switches were on on the previous day, the corresponding values of $p$ for this day are $1/4$, $2/4$ and $3/4$, respectively. Thus the transition matrix is
$$
\pmatrix{
\left(\frac34\right)^2&2\cdot\frac14\cdot\frac34&\left(\frac14\right)^2\\
\left(\frac24\right)^2&2\cdot\frac24\cdot\frac24&\left(\frac24\right)^2\\
\left(\frac14\right)^2&2\cdot\frac34\cdot\frac14&\left(\frac34\right)^2\\
}=\frac1{16}\pmatrix{9&6&1\\4&8&4\\1&6&9}\;.
$$
A: IMO the given information is sufficient.
2 bulbs, 2 states - ON & OFF each. That means total 4 states are possible per day:
OFF-OFF, ON-OFF,OFF-ON, ON-ON. Therefore, the transition matrix will be a 4*4 one.
Given that
$P(any  one switch=open next day)= \frac {(1+ number of on switches during previous day)}{4}$
Therefore, the Transition probability matrix will be as follows.
$00$  $01$    $10$  $11$
$00$     $\frac{9}{16}$  $\frac{3}{16}$ $\frac{3}{16}$        $\frac{1}{16}$
$01$     $\frac{4}{16}$  $\frac{4}{16}$ $\frac{4}{16}$        $\frac{4}{16}$
$10$     $\frac{4}{16}$  $\frac{4}{16}$ $\frac{4}{16}$        $\frac{4}{16}$
$11$    $\frac{1}{16}$  $\frac{3}{16}$ $\frac{3}{16}$        $\frac{9}{16}$
Hope this helps.
A: Just to add on to the last part of this question. The transition matrix with rows (left to right) and columns (top to bottom) representing [0 switch on, 1 switch on, 2 switch on]
$$\begin{pmatrix}0.5625 & 0.375 & 0.0625\\\ 0.25 & 0.5 & 0.25\\\ 0.0625 & 0.375 & 0.5625\end{pmatrix}$$
Using the conditions that a unique $\vec{\pi}$ satisfies the following equation: $\vec\pi=\pi P_{ij}$ and fulfils the condition $\sum_i{\pi_i}$ in $\vec\pi$.
We get $$\vec\pi=\left[\frac{2}{7},\frac{3}{7},\frac{2}{7}\right]$$
So to answer the question, the fraction of days where both switches are on is $\frac{2}{7}$. And the fraction of days where both switches are off is similarly $\frac{2}{7}$.
