0
$\begingroup$

I'm a bit stuck even setting up the following problem as a Markov Chain. I am thinking of some sort of random walk but the fact that the probability of heads depends on the person tossing the coin is confusing me.

A coin-tossing game is played by two players, A1 and A2. Each player has a coin and the probability that the coin tossed by player Ai comes up heads is $p_i$, where $0 < p_i < 1, i = 1, 2.$

The players toss their coins according to the following scheme: A1 tosses first and then after each head, A2 pays A1 one pound and A1 has the next toss, while after each tail, A1 pays A2 one pound and A2 has the next toss.

Define a Markov chain to describe the state of the game.

Find the probability that the game ever returns to a state where neither player has lost money.

$\endgroup$
0

1 Answer 1

0
$\begingroup$

One way to formulate the process is as follows.

Let state 0 mean A1 is tossing. Let state 1 mean A2 is tossing. Then,

$p_{00} = \mathbb P(Heads|A1)$

$p_{11} = \mathbb P(Heads|A2)$

  • Then, $p_{01}$ and $p_{10}$ are the complement events of $p_{00}$ and $p_{11}$, respectively. (Using your notation, $p_1$, the probability of heads when A1 is tossing, is the same value as $p_{11}$, the probability of transitioning to state 1 given that you started in state 1). We know how to compute probabilities for complement events $(\mathbb{P}(E) = 1 - \mathbb{P}(E^c))$. The 2x2 transition matrix is easily filled in.

  • You may be able to formulate the states of the chain to represent the amount of money held by each player/the sum/the difference of their holdings, depending on what information you will want to draw after setting up the matrix.

  • Now, "the probability that the game ever returns to a state where neither player has lost money" reminds me of the probability that the game is "zero-sum" for both players. I am inclined to say that since you have not described how the game "ends," it is therefore guaranteed that, after an indefinite amount of time passes, each player will end up at a point where they have lost all of their money, as the chain is recurrent (think about limits). If, for example, the game stopped when both players reached x quantity of money, the probability would be different. You may want to refer to the common gambler's ruin problem for more details. It turns out that the probability of winning or losing depends heavily on the probability of success, at least in the simple, most common example.

  • https://en.wikipedia.org/wiki/Gambler%27s_ruin#N-player_ruin_problem

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .