Markov chain - probability The biologist observed the bugs population in time. He found out that every bug lives for three years. The first year survives with a probability of 1/2. Those who survive the first year will survive the second year with a probability of 1/3. In the third year, each bug will "create" 6 new bugs and then he will die.
I have to construct transition matrix somehow. At first I tried this
$$\begin{pmatrix}
\begin{array}{ccc} 
0 & 0 & 6\\ 
1/2 & 0 & 0\\ 
0 & 1/3 & 0
\end{array}
\end{pmatrix}$$
where each $i$-th column represents a bug in a $i$-th year of age.
The initial state is 3000 bugs in the age of one (just born) and if I multiply the matrix with the vector $x_0=(3000,0,0)^T$, it works exactly as I want.
But it does not meet criterion of stochastic matrix that the sum of elements in a column must equal to 1. Additionally, it has strange Eigenvalues. So I am sure it's pretty bad matrix.
Could you give me a hint how to construct the matrix, please?
 A: Is this a Markov chain?
Formally speaking (but without getting overly rigorous), a discrete-time Markov chain is a sequence $(X_n)$ of random variables which satisfy the property that
$$\DeclareMathOperator{Pr}{Pr}\Pr(X_{n+1} = x_{n+1} \mid X_{n} = x_n, X_{n-1}=x_{n-1}, \dotsc, X_1 = x_1) = \Pr(X_{n+1} = x_{n+1} \mid X_{n}).$$
In other words, if you think of $X_n$ as the state of some system after $n$ (discrete) units of time have elapsed, the probability that the system is in some particular state at time $n+1$ depends only on the state of the system at time $n$.
In order to describe a Markov chain, it is useful to first describe the possible states.  In the problem described here, the possible states can be represented by $3$-tuples which describe the populations of "zero year olds", "one year olds", and "three year olds".  For example, the tuple
$$ x = (1000, 500, 100) $$
represents a state in which there are $1000$ newly hatched bugs, $500$ bugs which are one year old, and $100$ bugs which are two years old.  Notice that the state space (the collection of all possible states) is countably infinite—indeed, it is the set $\mathbb{N}_0^3$, where $\mathbb{N}_0$ is the set of natural numbers (including zero).  This is a relatively large state space, which complicates things a little.
Having described the state space, the next step is to describe the transition probabilities from one state to another.  Unfortunately, this is probably not tractable in the current problem.  Even in a relatively simple case, e.g.
$$ x_{n} = (10,10,10), $$
the "next" state could be any of $121$ possible states:
$$ x_{n+1} \in \bigl\{ (30,m,n) : m,n \in \{0,1,2,3,4,5,6,7,8,9,10\}\bigr\}. $$
Moreover, even if one has a practical way of computing transition probabilities, the transition "matrix" $T$ is going to have an infinite number of rows and columns.  Indeed, it is more appropriate to think of $T$ as a linear operator.  This operator takes a sequence $$p= (p_{ijk})_{i,j,k \in \mathbb{N_0}}$$ as input, where $p_{ijk}$ is the probability that the system is in state $(i,j,k)$, and outputs another sequence $Tp$.  While there is a theory of operators on these kinds of spaces, it is not likely to be very helpful in this situation, and would require a much deeper examination of probability theory and functional analysis to go in this direction.
So, while it is (in principle) possible to model the problem by describing the possible states and the transition probabilities between those states, this is the Wrong Approach™ for this problem.
How do we answer the question being asked?
Fortunately, the question being asked is not "what is the probability that the population of bugs is $(i,j,k)$ at time $n$?", but rather, "what is the expected population of each age group of bugs at time $n$?"  This is a much more tractable question.
Abusing notation and language a little (but in a way which can be made rigorous if need be), if the state at time $n$ is thought of as a vector $x = \langle x_1, x_2, x_3\rangle$, then the expected state at time $n+1$ is given by
$$ \underbrace{\pmatrix{ 0 & 0 & 6 \\ 1/2 & 0 & 0 \\ 0 & 1/3 & 0}}_{=:A} \pmatrix{ x_1 \\ x_2 \\ x_3} = Ax.$$
Roughly speaking, the population of one year olds at time $n+1$ is expected to be half the population of zero year olds at time $n$ (half of the zero year olds survive to become one year olds, and none of the one or two year olds somehow become one year olds).  Similar reasoning applies to the zero and two year olds, which gives the matrix $A$.
Assuming that the initial population is $x = \langle x_1, x_2, x_3 \rangle$, then the expected population at time $n=1$ is $Ax$, the expected population at time $n=2$ is $A^2x$, and so on.  It follows that the expected population at time $n$ is
$$ A^n x. $$
While this is a bit out of the scope of this question, if one actually wants to compute $A^nx$, it is helpful to diagonalize $A$, i.e. write it in the form
$$ A = PDP^{-1},$$
where $P$ is a unitary matrix and $D$ is a diagonal matrix given by
$$ D = \pmatrix{\lambda_1 & 0 & 0 \\ 0 & \lambda_2 & \\ 0 & 0 & \lambda_3}. $$
The $\lambda_j$ are the eigenvalues of $A$.  The entries of $P$ (and $P^{-1}$) are a matter of computation—presumably, anyone asking this question should be capable of performing these computations.  It then follows that
$$ A^n = P D^n P^{-1} = P \pmatrix{\lambda_1^n & 0 & 0 \\ 0 & \lambda_2^n & \\ 0 & 0 & \lambda_3^n} P^{-1}, $$
which gives a viable method for determining the expected state $A^nx$ at time $n$, given that the system is initially in state $x$.
