Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm reading a paper that jumps from eq (1) to eq (2) without describing the steps taken. I'd like to understand how/why/what steps allow the transform. (My linear algebra is a bit rusty...)

(1) $\boldsymbol{V}^\pi = \boldsymbol{R} + \gamma \boldsymbol{P}_{a_1} \boldsymbol{V}^\pi$


(2) $\boldsymbol{V}^\pi = (\boldsymbol{I} - \gamma \boldsymbol{P}_{a_1})^{-1} \boldsymbol{R}$


  • $\boldsymbol{V}^\pi$ is a vector
  • $\boldsymbol{R}$ is a vector
  • $\boldsymbol{P}_{a_1}$ is an N-by-N matrix
  • $(\boldsymbol{I} - \gamma \boldsymbol{P}_{a_1})$ is guaranteed to be invertible due to the nature of the problem.

The superscript ($\pi$) and subscript ($a_1$) may be safely ignored as they identify particular instances of the vector and matrix. I include them for completeness and just in case I'm wrong about them being ignorable. Also, if it helps for context, the paper is dealing with Markov Decision Processes (MDP), and these equations are related.

I'm assuming the steps are simple since the author skips them, so this question is hopefully easy to answer. Thanks!

share|cite|improve this question
Those are the exact equations in the paper. – ɲeuroburɳ May 22 '12 at 15:25
up vote 1 down vote accepted

shift the $γP_{a1}V^π$ term to the other side, factor out the $V^\pi$ (to the right, remember) and apply the appropriate inverse matrix to both sides.

share|cite|improve this answer
Heh... I think I see it now. Just to verify, you're saying step 1 is $V^\pi - \gamma P_{a_1} V^\pi = R$. Step 2 is $(I - \gamma P_{a_1}) V^\pi = R$. Step 3 multiply both sides by $(I - \gamma P_{a_1})^{-1}$. – ɲeuroburɳ May 22 '12 at 15:30
exactly. Aside from remembering that matrix multiplication is non-commutative, you don't need any linear algebra here per se. – Robert Mastragostino May 22 '12 at 15:39
Thanks dude! Exactly what I needed. – ɲeuroburɳ May 22 '12 at 15:46

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.