This question was originally posted here, but as the Math.SE community is more active I provide an extended version of the post here.

I am working through the basic examples of the stochastic optimization models in the book by McCandless (2008): The ABCs of RBCs, pp. 71 - 75 [RBC stands for `real business cycle' model in the economists' jargon]

A Standard Stochastic Dynamic Programming Problem

In what follows next, I assume that the domain of the variables and the range of the functions all belong to $\mathcal{R}_0^+$ and I assume there are no corner solutions. Here is a formulation of a basic stochastic dynamic programming model: \begin{equation} y_t = A^t f(k_t) \end{equation}

\begin{equation} A^t = \cases{A_1 \text{ with probability } p \\ A_2 \text{ with probability } (1 - p) } \end{equation}

\begin{equation} k_{t+1} = A^tf(k_t) + (1 - \delta)k_t - c_t \end{equation}

With an agent maximizing the expected utility function $u(c_t)$, which is concave, monotone strictly increasing in $c_t$: \begin{equation} E_t \sum_{t}^\infty \beta^t u(c_t) \end{equation}

Substituting consumption from the previous equation and using the recursive formulation of the problem gives the following problem: \begin{equation} V(k_t, A^t) = \max_{k_{t+1}} \left[u(A^tf(k_t) + (1 - \delta)k_t - k_{t+1}) + \beta E_t V(k_{t+1},A^{t+1})\right] \end{equation}

Then McCandless proceeds saying that the algorithm to solve to the problem is almost identical to the deterministic case. One finds the first-order conditions (a derivative of the value function with respect to $k_{t+1}$) for the control variables, then does the same for $k_t$ and applies the Envelope theorem to get an analytical solution.

(1) Find the first order conditions for the control variables (period $t+1$): \begin{equation} \frac{\partial V_t}{\partial k_{t+1}} = - u_{k_{t+1}}(A^tf(k_t) + (1 - \delta)k_t - k_{t+1}) + \beta E_t V_{k_{t+1}}(k_{t+1},A^{t+1}) \end{equation}

To go around $V_{k_{t+1}}(k_{t+1},A^{t+1})$ he applies the results from Benveniste and Scheinkman

(2) Find the derivatives for the value function for the control variables of the previous time $t$ (which are the state variables in the case):

\begin{equation} \partial V_t / \partial k_{t} = A^t (f_{k_t}(k_t) - \delta) u_{k_t}(A^tf(k_t) + (1 - \delta)k_t - k_{t+1}) \end{equation}

And obtain the value of $\partial V_t / \partial k_{t}$ by simply `updating' it by one period (refering to Benveniste and Scheinkman):

\begin{equation} \partial V_t / \partial k_{t+1} = A^t (f_{k_{t+1}}(k_{t+1}) - \delta) u_{k_{t+1}}(A^tf(k_{t+1}) + (1 - \delta)k_{t+1} - k_{t+2}) \end{equation}

Then you can impute the above equation into the first-order conditions and derive a steady-state solution with the Euler equation. Model written, implications discussed, paper submitted. Profit.

A Lottery Augmented Version

Now I want to investigate a little different case. Take the very same model but introduce another variable. Now allow agent to save some of his/her income in every period (call it $s_t$ for a security) to accumulate in a variable $l$: \begin{equation} l_{t+1} = l_t + s_t \end{equation}

And I assume here that l_{t+1} is the control variable because in terms of accounting it does not matter, which one to pick $l_{t+1}$ or $s_t$ but it helps to derive the optimality conditions later on.

And the $l_t$ enters the problem through the variable $A^t$: \begin{equation} A^t = \cases{A_1 \text{ with probability } p \\ A_2l_{t} \text{ with probability }(1 - p)} \end{equation}

The main difference is easy to see if one writes the equation for income in a period $t$ explicitly opening up the expectation sign: \begin{equation} y_t = pA_1f(k_t) + (1-p)A_2l_tf(k_t) \end{equation}

In this case we have a deterministic control variable $l_t$ "turning on" when a certain event happens (as if you would win in a lottery, which increases your income by the factor of how much you invested in it; yes the example makes little sense but I am interested in the principle itself).

The Question: Would the previous algorithm described by McCandless give the right solution as well?

P.S. If someone could point me a paper with an example model, which is very close to the one I described, that would be helpful.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.