1
$\begingroup$

This question was originally posted here, but as the Math.SE community is more active I provide an extended version of the post here.

I am working through the basic examples of the stochastic optimization models in the book by McCandless (2008): The ABCs of RBCs, pp. 71 - 75 [RBC stands for `real business cycle' model in the economists' jargon]

A Standard Stochastic Dynamic Programming Problem

In what follows next, I assume that the domain of the variables and the range of the functions all belong to $\mathcal{R}_0^+$ and I assume there are no corner solutions. Here is a formulation of a basic stochastic dynamic programming model: \begin{equation} y_t = A^t f(k_t) \end{equation}

\begin{equation} A^t = \cases{A_1 \text{ with probability } p \\ A_2 \text{ with probability } (1 - p) } \end{equation}

\begin{equation} k_{t+1} = A^tf(k_t) + (1 - \delta)k_t - c_t \end{equation}

With an agent maximizing the expected utility function $u(c_t)$, which is concave, monotone strictly increasing in $c_t$: \begin{equation} E_t \sum_{t}^\infty \beta^t u(c_t) \end{equation}

Substituting consumption from the previous equation and using the recursive formulation of the problem gives the following problem: \begin{equation} V(k_t, A^t) = \max_{k_{t+1}} \left[u(A^tf(k_t) + (1 - \delta)k_t - k_{t+1}) + \beta E_t V(k_{t+1},A^{t+1})\right] \end{equation}

Then McCandless proceeds saying that the algorithm to solve to the problem is almost identical to the deterministic case. One finds the first-order conditions (a derivative of the value function with respect to $k_{t+1}$) for the control variables, then does the same for $k_t$ and applies the Envelope theorem to get an analytical solution.

(1) Find the first order conditions for the control variables (period $t+1$): \begin{equation} \frac{\partial V_t}{\partial k_{t+1}} = - u_{k_{t+1}}(A^tf(k_t) + (1 - \delta)k_t - k_{t+1}) + \beta E_t V_{k_{t+1}}(k_{t+1},A^{t+1}) \end{equation}

To go around $V_{k_{t+1}}(k_{t+1},A^{t+1})$ he applies the results from Benveniste and Scheinkman

(2) Find the derivatives for the value function for the control variables of the previous time $t$ (which are the state variables in the case):

\begin{equation} \partial V_t / \partial k_{t} = A^t (f_{k_t}(k_t) - \delta) u_{k_t}(A^tf(k_t) + (1 - \delta)k_t - k_{t+1}) \end{equation}

And obtain the value of $\partial V_t / \partial k_{t}$ by simply `updating' it by one period (refering to Benveniste and Scheinkman):

\begin{equation} \partial V_t / \partial k_{t+1} = A^t (f_{k_{t+1}}(k_{t+1}) - \delta) u_{k_{t+1}}(A^tf(k_{t+1}) + (1 - \delta)k_{t+1} - k_{t+2}) \end{equation}

Then you can impute the above equation into the first-order conditions and derive a steady-state solution with the Euler equation. Model written, implications discussed, paper submitted. Profit.

A Lottery Augmented Version

Now I want to investigate a little different case. Take the very same model but introduce another variable. Now allow agent to save some of his/her income in every period (call it $s_t$ for a security) to accumulate in a variable $l$: \begin{equation} l_{t+1} = l_t + s_t \end{equation}

And I assume here that l_{t+1} is the control variable because in terms of accounting it does not matter, which one to pick $l_{t+1}$ or $s_t$ but it helps to derive the optimality conditions later on.

And the $l_t$ enters the problem through the variable $A^t$: \begin{equation} A^t = \cases{A_1 \text{ with probability } p \\ A_2l_{t} \text{ with probability }(1 - p)} \end{equation}

The main difference is easy to see if one writes the equation for income in a period $t$ explicitly opening up the expectation sign: \begin{equation} y_t = pA_1f(k_t) + (1-p)A_2l_tf(k_t) \end{equation}

In this case we have a deterministic control variable $l_t$ "turning on" when a certain event happens (as if you would win in a lottery, which increases your income by the factor of how much you invested in it; yes the example makes little sense but I am interested in the principle itself).

The Question: Would the previous algorithm described by McCandless give the right solution as well?

P.S. If someone could point me a paper with an example model, which is very close to the one I described, that would be helpful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.