4
$\begingroup$

I have a machine. It has two states, broken or working. If it is working, then it will be broken with probability $q=0.1$. If the machine is working, I will make \$1000 dollar a day. If it is broken, then repairman will charge charge me \$ $200/(1-p) $ a day to repair. He will fix the machine with probability p. Assume the transition from broken to working (and wise versa) is independent. Find p that maximize the expected profit.

Attempt:

2 state markov chain. Let state 0 be working, and let state 1 be broken. The state transition matrix is:

\begin{pmatrix} 1-q & q \\ p & 1-p \\ \end{pmatrix}

The steady state distribution is calculated by (omit showing my calculation process here since it is well-known) : $\pi_0=q/(p+q), \pi_1=p/(p+q)$

The expected profit is: $1000*\pi_0-200/(1-p)*\pi_1$. (Do you think this is correct)?

$\endgroup$
4
  • 1
    $\begingroup$ You have the right idea, but you made a mistake in computing $\pi_0, \pi_1$ (I believe you swapped the $p$s and $q$s). The equation should be $$\pi_{working}q= \pi_{broken}p$$. $\endgroup$
    – Michael
    Feb 11, 2016 at 2:30
  • 1
    $\begingroup$ To verify, you can get the same answer from renewal theory: $$\mbox{time avg profit} = \frac{1000E[working] -\frac{200}{1-p}E[broken]}{E[working] + E[broken]}$$ where $E[working]$ is the average duration of a working interval, and $E[broken]$ is the average duration of a broken interval. $\endgroup$
    – Michael
    Feb 11, 2016 at 2:34
  • $\begingroup$ In that case, $E[working]/(E[working]+E[broken])$ would be the stationary distribution? $\endgroup$
    – randy
    Feb 11, 2016 at 5:57
  • 1
    $\begingroup$ Yes, $\pi_{working} = \frac{E[working]}{E[working]+E[broken]}$. $\endgroup$
    – Michael
    Feb 11, 2016 at 6:39

1 Answer 1

1
$\begingroup$

You messed it a bit. It should be: $\pi_0=\frac{p}{(p+q)}, \pi_1=\frac{q}{(p+q)}$ So the profit is: $$-200\frac{0.1}{(1-p)(p+0.1)}+1000\frac{p}{p+0.1}$$ So you have to find the optimum between 0 and 1. The first derivative is: $$-20\frac{2p-0.9}{(1-p)^2(p+0.1)^2}+1000\frac{0.1}{(p+0.1)^2}$$ You put it equal to zero and you get the sollution, which is: $p=\frac{12-\sqrt{26}}{10} \approx 0.690098049$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .