Value function of Bellman Equation Two problems coming from Optimization in Economics (Permanent Income Hypothesis)
Q1. Why Bellman equation is $$V(s)=r(s)+\beta\sum_{s'} p(s'\mid s)V(s')$$
instead of $$V(s)=r(s)+\beta\sum_{s'} p(s'\mid s)r(s')$$
where $r$ is the payment given state?
Q2. In deterministic case, maximization of $$\sum_{t}\beta^tu(c_t)$$ subject to constraint $$a_{t+1}=\frac{1}{\beta}(a_t+y_t-c_t)$$ gives Euler equation $$\frac{1}{\beta}=\frac{u'(c_{t+1})}{u'(c_t)}$$ by first order condition of Lagrangian
However, in stochastic case, do we have $$\frac{1}{\beta}=\mathbb{E}_t[\frac{u'(c_{t+1})}{u'(c_t)}]$$?
Is there actually an stochastic Lagrangian formulation and we  set $$\lambda_t$$ depend on $t$ when doing optimization? Intuitively suppose we have income process $$y_{t+1}=y_t+\epsilon_t$$, it's really hard to say how much I will save based on the size of the shock. 
If the formulation is hard we can only use the method in Q1
 A: To Q1. $r(s)$ is the payment given that you are at state $s$. But this is only the payment for one period and you have it already in front of the summation. You need the $V(s)$ because you are considering the whole sequence of events that will follow in the future, given that you are now at state $s$. $V(s)$ stands for the whole future after reaching state $s$, including of course future payoffs $r(\cdot)$, while $r(s)$ is only the payoff of the first (current) period.
As an example, consider the case where you have only two states, $s,s'$. Then the Bellman equation is: $$V(s)=r(s)+βp(s\mid s)V(s)+βp(s'\mid s)V(s')$$
and it reads:


*

*"$V(s)=$": the best you can do when starting from state $s$ is equal to 

*"$r(s)$": the payoff at state $s$

*"$+\beta p(s\mid s)V(s)$": plus the (discounted) best that you can do from the next period, given that you will start from $s($hence $V(s)$ and not $r(s))$, weighted of course with the probability to stay at $s$ given that you are now at $s$. 

*"$+\beta p(s'\mid s)V(s')$": plus the (discounted) best that you can do from the next period, given that you will start from $s'($hence $V(s')$ and not $r(s'))$, weighted of course with the probability to move to $s'$ given that you are now at $s$.

