How to determine transient and recurrent states in a stochastic matrix?

I am wondering if there is a general formula for doing this.

Consider my favorite simple example: Consider stochastic matrix $P$ with entities $p_{11}=1/3$, $p_{22}=5/6$, $p_{12}=2/3$, $p_{21}=1/6$.

Is there a way to find out what are the recurrent and transient states without having to find a general formula for $P^n$?

-
Yes: both states are recurrent. Do you know what a recurrent state and a transient state are? – Did Jan 26 '12 at 14:44
Yes. A state is transient if there is a nonzero probability that starting from that state you may never transition back to that state. But how do you show this?? – Brian Jan 26 '12 at 16:00
Either from first principles, showing the hitting time of i starting from 3-i is almost surely finite by computing its distribution. Or from the elementary theory of Markov chains, which ensures that every irreducible finite Markov chain is recurrent. So, it all depends on what you know and what you don't--a subject on which you are totally silent. – Did Jan 26 '12 at 18:20