Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have the following optimal control problem, which can be considered in a two period discrete time setting. It has this generic form.

Let $w: \mathbb{R} \rightarrow \mathbb{R}$ be a given function, modeling terminal reward (in my example in mind, $w$ is negative, bounded, increasing, and concave). At time $0$, the state of the system is known, and equal to $x$. The controller can choose a control $t \in [0,\infty )$ such that the state of the system at time $1$ is $f(t,x,Z)$, where $Z$ is a standard random variable. The cost of using strategy $t$ is $g(t,x)$. The goal is to calculate

$\hat{w}(x) = \underset{t}{min} \ \left( g(t,x) + e^{-\lambda t} E [w(f(t,x,Z))] \right),$

and find the optimal $t$. Is there a reference for explicit/computable solutions to this problem? Subject to certain restrictions on w,f,g?

share|cite|improve this question
I would consider this to be a problem of minimizing an expected value (ie, $\min_t E[g(t,x) + e^{-\lambda t} w(f(t,x,Z))]$) rather than an optimal control problem. The solution techniques depend on how you evaluate the expectation. – copper.hat Sep 25 '12 at 21:19
can you elaborate a little bit on your last sentence? – quasi Sep 25 '12 at 21:20
Are you using Monte Carlo or do you have a way of explicitly evaluating the expectation? If the former, you might look up work by Georg Pflug, if the latter, then this is an 'ordinary' optimization problem. – copper.hat Sep 25 '12 at 21:21
The expectation can't be computed analytically. How can this be solved using monte carlo? Is there another numeric method for solving it without probability? – quasi Sep 25 '12 at 21:26
I'm not sure how you can avoid probability if you have an expectation that cannot be computed explicitly? – copper.hat Sep 25 '12 at 21:27

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.