Sign up ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Can someone please explain the difference between dynamic optimization via the Bellman equation and dynamic optimization via Pontryagin's maximization principle?


share|cite|improve this question

2 Answers 2

up vote 8 down vote accepted

The Bellman principle poses an optimization problem using a nonlinear 1st order partial differential equation - the object being optimized is a function. Pontryagin's maximum principle poses the same problem using a form of the calculus of variations - the optimized object is a curve.

This implies in technical differences and different conditions for existence of solutions, which are discussed in the literature. The optimal curve given by the maximum principle is a Cauchy characteristic of the Hamilton Jacobi Bellman partial differential equation, assuming technical conditions such that both exist.

Bellman's method was originally formulated for discrete-time systems, and extended to continuous time ones. The maximum principle is mostly studied in the continuous time framework.

share|cite|improve this answer
Thanks. Also, I looked through Pontryagin's book the other day and he clearly addresses this difference. It is interesting to note that the Bellman approach does not actually constitute a mathematical solution, but rather a "good heuristic" (in Pontryagin's words). The maximum principle on the other hand is an actual mathematical solution. I wonder why this detail is never mentioned when the Bellman equation is being taught. –  ben Oct 27 '11 at 19:38
I respectfully submit that Pontryagin's words should be seen in the light of the Cold War, and perhaps of his personal prejudices, rather than from the mathematical viewpoint. –  Pait Mar 9 '12 at 14:01

In these optimzation problems one needs to find a trajectory - that a curve or a sequence. Pontryagin's theory produces just this. Bellman's equation is a partial differential equation that is "woven" of Pontryagin curves - hence it looks like an overkill, unnecessary complication. I can not figure out why it is so popular. I found that Bellman's equation is used to derive in different way (almost) the same equations (they differ by the independent variable) that follow from Pontryagin's theory and then they are solved as if Bellman was never born. Bellma's principle of optimality is very charming, intuitional and handy, but it seems only a matter of taste which method one uses. What's more interesting is that the method of Lagrange's multipliers may also be used and it leads to the same conclusions as the two methods. All the three methods work well for discrete and continuous time. It seems to me that Bellman's method seems to require fewer prerequisities, that is, the teachers that use Bellmann's method do not bother to state the technical assumptions, and their students love this, intricacies of Pontyagin's theory are perhaps overwhelming.

share|cite|improve this answer
The Bellman equation is useful precisely because it describes all of the curves. This is interesting when the boundary conditions are themselves not known a priori - perhaps they are also quantities to be optimized. This happens for instance in the study of state estimators (the Kalman filter of linear control theory). So no, it is not overkill. Bellman's method needs more technical smoothness assumptions. When they are satisfied, it is indeed mor intuitive and easier to understand. –  Pait Dec 26 '13 at 15:31

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.