Suppose we are given a stochastic dynamic programming problem. $$\max E\sum_{t=0}^T F(t,X_t, X_{t+1}(X_t,V_{t}),V_{t})$$ Where $V_t$ is a random variable, correlated possibly with $V_{t-1}$. In this problem, we assume that the "optimizer" can choose $X_1$ while knowing the outcome of $V_0$, choose $X_{2}$ while knowing the outcome of $V_0,V_{1}$, and so forth. Because of this, as Sydaeter & Hammond state, the optimizer can assume at time $t$, that at time $t+1$ he will choose $X_{t+1}$ based on his knowledge of $X_t$ and $V_t$.

In other words, $X_{t+1}=X_{t+1}(X_t,V_t)$.

Sydaeter & Hammond ("Further mathematics for economic analysis"), state that we can formulate a "stochastic Euler Equation" (see below).

enter image description here

However, My intuition is that this is incorrect. My intuition is that it is impossible to derive a stochastic Euler equation, for the simple reason that every $X_{t+k}$ depends on $X_t$, through an iterative application of the function $X_{t+1}(X_t, V_t)$. If that is the case, then shouldn't the choice of $X_t$ affect the reward function $F$ through its effect on all $X_{t+k}$'s, so that the Euler equation is an infinite sum of derivatives?

  • $\begingroup$ Your question is very interesting. Indeed I questioned myself if taking the derivatives was appropriate. I don't know if adding an example may clarify the idea that you have explained in a more general setting. $\endgroup$ – deps_stats Dec 12 '17 at 5:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.