Can someone please explain the difference between dynamic optimization via the Bellman equation and dynamic optimization via Pontryagin's maximization principle?
The Bellman principle poses an optimization problem using a nonlinear 1st order partial differential equation - the object being optimized is a function. Pontryagin's maximum principle poses the same problem using a form of the calculus of variations - the optimized object is a curve.
This implies in technical differences and different conditions for existence of solutions, which are discussed in the literature. The optimal curve given by the maximum principle is a Cauchy characteristic of the Hamilton Jacobi Bellman partial differential equation, assuming technical conditions such that both exist.
Bellman's method was originally formulated for discrete-time systems, and extended to continuous time ones. The maximum principle is mostly studied in the continuous time framework.
In these optimzation problems one needs to find a trajectory - that a curve or a sequence. Pontryagin's theory produces just this. Bellman's equation is a partial differential equation that is "woven" of Pontryagin curves - hence it looks like an overkill, unnecessary complication. I can not figure out why it is so popular. I found that Bellman's equation is used to derive in different way (almost) the same equations (they differ by the independent variable) that follow from Pontryagin's theory and then they are solved as if Bellman was never born. Bellma's principle of optimality is very charming, intuitional and handy, but it seems only a matter of taste which method one uses. What's more interesting is that the method of Lagrange's multipliers may also be used and it leads to the same conclusions as the two methods. All the three methods work well for discrete and continuous time. It seems to me that Bellman's method seems to require fewer prerequisities, that is, the teachers that use Bellmann's method do not bother to state the technical assumptions, and their students love this, intricacies of Pontyagin's theory are perhaps overwhelming.