Optimal Control example from Wikipedia I am interested in Optimal Control:
https://en.wikipedia.org/wiki/Optimal_control
and I am reading the discrete example at the bottom of the Wikipedia page:

where they give the solution as:

I have some questions (from top red annotations to bottom):


*

*Where does ($\lambda_{t+1} u_t$) come from?

*How did they equate negative the differential, to $-(\frac{u_t}{x_t})^2$ ?

*What does it mean using the initial and turn T conditions?

 A: 1) By definition of Hamiltonian, $H(x,u,\lambda,t)$ is equal to cost function plus Lagrange multiplier times into constraint. In discrete time system, this becomes equal to $\lambda_{t+1}$ because of aesthetic reason and numerous simplification to solution. One of the reason is because your discrete time system model is $x(t+1) = f(x_t,u_t)$, where $t$ can only take discrete values. In the above example, firstly its not the Hamiltonian of the entire system because it should also include the summation but is the Hamiltonian at time $t$ only. Therefore Hamiltonian is mentioned the way you have written.
2) Simply take partial derivative with respect of $x_t$ on Hamiltonian and multiply it with negative will yield the result. Why is it equal to left hand side is by the Calculus of variation theory in optimal control. Its part of necessary condition for optimality.
3) The equation that you solved are actually differential equations and differential equations need boundary values to be solved. Initial and Final time are boundary conditions that you will need to solve to find the exact solution. Otherwise you will have general constant of integration.
