Use linear quadratic regulator to minimize output error I would like to create an Infinite-horizon, continuous-time LQR with a cost functional defined as
$$J = \int_{0}^\infty \left( e^T Q e + u^T R u \right) dt$$
where e is the states' error $x-x_d$, but I have trouble concluding to the appropriate Ricatti equation. My problem is that since $x_d$ is a function of time the derivative of the Lyapunov function $V=e^T P e$ leads to a term with $\dot x_d$ . Is this problem solvable? Any ideas?
 A: Consider the linear time invariant system
$$\dot x(t)=Ax(t)+Bu(t),$$
where$~x(t)\in\mathbb{R}^n~$is the state vector and $~u(t)\in\mathbb{R}^m~$is the input vector and 
$$A:\mathbb{R}^n\rightarrow\mathbb{R}^{n\times n};B:\mathbb{R}^n\rightarrow\mathbb{R}^{m\times n}.$$
The following cost function is to be minimized:
$$J=\frac{1}{2}\int_{0}^{\infty}((x(t)-x_{d}(t))^{T}Q(x(t)-x_{d}(t))+u^{T}(t)R{u}(t))dt,$$
where $~x_{d}(t)\in\mathbb{R}^n~$is the desired output.
Then considering error as $~e(t)=x(t)-x_{d}(t)~$ and $~\dot e(t)=\dot x(t)-\dot x_{d}(t)~$, the system's equation turns into:
$$\dot e(t)=Ae(t)+Bu(t)-\dot x_{d}(t)+Ax_{d}(t),$$
where the terms $~g(t)=-\dot x_{d}(t)+Ax_{d}(t)~$ are functions of time, makes the system non-homogenous.
The Hamiltonian is shaped as:
$$H(x(t),x_{d}(t),u(t),\lambda (t),t)=\frac{1}{2}((x(t)-x_{d}(t))^{T}Q(x(t)-x_{d}(t))+u^{T}(t)R{u}(t))+\lambda^{T}(t)(A(x(t)-x_{d}(t))+Bu(t)-\dot x_{d}(t)+Ax_{d}(t)).$$
The necessary condition results the control law
$$\frac{\partial H(x(t),x_{d}(t),u(t),\lambda (t),t)}{\partial u(t)}=0\Rightarrow u(t)=-R^{-1}B^{T}\lambda (t).$$
Setting $$\lambda (t)=K(t)(x(t)-x_{d}(t)),$$
input law refomred as$$u(t)=-R^{-1}B^{T}K(t)(x(t)-x_{d}(t)).$$
Next optimality condition $$\frac{\partial H(x(t),x_{d}(t),u(t),\lambda (t),t)}{\partial (x(t)-x_{d}(t))}=-\dot \lambda(t),$$
results
$$Qe(t)+A^{T}K(t)e(t)=-\dot K(t)e(t)-K(t)\dot e(t),$$
which is rewritten as
$$(A^{T}K(t)+K(t)A-K(t)BR^{-1}B^{T}K(t)+Q)e(t)+K(t)g(t)=-\dot K(t)e(t).$$
If we tend $~t\rightarrow \infty~$, then $~\dot K(t)\rightarrow 0~$ and optimal gain turns to steady state gain as $~K(t)=K_{ss}~$ and the new equation to:
$$(A^{T}K_{ss}+K_{ss}A-K_{ss}BR^{-1}B^{T}K_{ss}+Q)e(\infty)+K_{ss}g(\infty)=0.$$
It seems in order to have an answer, the non-homogenous term should tend to zero or in other words it must be damped by the system and controller over long time, with this assumption the Riccati equation then can be realized as:
$$A^{T}K_{ss}+K_{ss}A-K_{ss}BR^{-1}B^{T}K_{ss}+Q=0.$$
To prove the stability, Lyapunov function could be used as
$$V(t)=e^{T}(t)K_{ss}e(t),$$
which is non-negative then
$$\dot V(t)=\dot e^{T}(t)K_{ss}e(t)+e^{T}(t)K_{ss}\dot e(t),$$
$$\dot V(t)=(Ae(t)+Bu(t)-g(t))^{T}(t)K_{ss}e(t)+e^{T}(t)K_{ss}(Ae(t)+Bu(t)-g(t)),$$
$$\dot V(t)=e^{T}(t)(A^{T}K_{ss}+K_{ss}A-K_{ss}BR^{-1}B^{T}K_{ss}-K_{ss}BR^{-1}B^{T}K_{ss})e(t)-g^{T}(t)K_{ss}e(t)-e^{T}(t)K_{ss}g(t),$$
$$\dot V(t)=-(e^{T}(t)(Q+K_{ss}BR^{-1}B^{T}K_{ss})e(t)+g^{T}(t)K_{ss}e(t)+e^{T}(t)K_{ss}g(t)),$$
turns out to be negative, if the term $~g^{T}(t)e(t)>0~$.
Good luck.
