6
$\begingroup$

I have a machine. It has two states, broken or working. If it is working, then it will be broken with probability $q=0.1$. If the machine is working, I will make \$1000 dollar a day. If it is broken, then repairman will charge charge me \$ $200/(1-p) $ a day to repair. He will fix the machine with probability p. Assume the transition from broken to working (and wise versa) is independent. Find p that maximize the expected profit.

Attempt:

2 state markov chain. Let state 0 be working, and let state 1 be broken. The state transition matrix is:

\begin{pmatrix} 1-q & q \\ p & 1-p \\ \end{pmatrix}

The steady state distribution is calculated by (omit showing my calculation process here since it is well-known) : $\pi_0=q/(p+q), \pi_1=p/(p+q)$

The expected profit is: $1000*\pi_0-200/(1-p)*\pi_1$. (Do you think this is correct)?

$\endgroup$
  • 1
    $\begingroup$ You have the right idea, but you made a mistake in computing $\pi_0, \pi_1$ (I believe you swapped the $p$s and $q$s). The equation should be $$\pi_{working}q= \pi_{broken}p$$. $\endgroup$ – Michael Feb 11 '16 at 2:30
  • 1
    $\begingroup$ To verify, you can get the same answer from renewal theory: $$\mbox{time avg profit} = \frac{1000E[working] -\frac{200}{1-p}E[broken]}{E[working] + E[broken]}$$ where $E[working]$ is the average duration of a working interval, and $E[broken]$ is the average duration of a broken interval. $\endgroup$ – Michael Feb 11 '16 at 2:34
  • $\begingroup$ In that case, $E[working]/(E[working]+E[broken])$ would be the stationary distribution? $\endgroup$ – randy Feb 11 '16 at 5:57
  • 1
    $\begingroup$ Yes, $\pi_{working} = \frac{E[working]}{E[working]+E[broken]}$. $\endgroup$ – Michael Feb 11 '16 at 6:39
0
$\begingroup$

You messed it a bit. It should be: $\pi_0=\frac{p}{(p+q)}, \pi_1=\frac{q}{(p+q)}$ So the profit is: $$-200\frac{0.1}{(1-p)(p+0.1)}+1000\frac{p}{p+0.1}$$ So you have to find the optimum between 0 and 1. The first derivative is: $$-20\frac{2p-0.9}{(1-p)^2(p+0.1)^2}+1000\frac{0.1}{(p+0.1)^2}$$ You put it equal to zero and you get the sollution, which is: $p=\frac{12-\sqrt{26}}{10} \approx 0.690098049$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.