# Using Time Homogenous Markov Chain to maximise profit

Suppose there is a company with 30 printing machines and they need to hire staff to operate the machines such that the profit is maximised. The model is that each machine is either 1. Idle state (not operating) 2. Setup state (Operator is working on readying it for printing) 3. Working state (machine is printing, operator not needed)

When a machine completes a print job (working state), it enters either setup state (if there is a free operator available) or Idle state (if no operators are free).

When an operator finishes Setup state of a machine, the machine enters Working state and the Operator either selects an Idle machine and start Setup or if no machines are available, the operator waits until next machine becomes available.

The length of each print job (in days) follows Exp(7) and length of time (in days) to set up a machine follows Exp(1) dist. Setup times and print times are independent.

While in working, the machine earns $$30/day and$$0 in other states. Each operator costs \$80 per day. The question is how to maximise profit, by mainly using Time Homogenous Markov chains and R coding