Conditions for optimal control Consider the following system:
$\text{minimize}_{u} \text{ } J = \int_0^\infty u^2 dt$
Subject to: $\dot{x} = Ax + Bu, \text{ } x(0) = x_0$
Where: $A = \begin{bmatrix} k & 1 \\ 0 & 1 \end{bmatrix}$, $B = \begin{bmatrix} 0 \\ 1 \end{bmatrix}$.
What are the conditions for $k$ such that the optimal control exists?
I can show that the Riccati-equation:
$A^TP + PA - PBB^TP = 0$ has only symmetric positive-definite solutions $P$ when $k>0$.
Can an optimal control exist if $P$ is not a symmetric positive-definite solution?
Any help appreciated!
 A: Consider there is a $P$ such that
$$A^T P+PA-PBB^TP=0$$
then the state feedback is $K=B^TP$ and the closed-loop state matrix would be:
$$A-BK=A-BB^TP$$
Needless to say, an optimal control has to stabilize the system. Which means the closed-loop state matrix must be Hurwitz. Now the non-trivial solutions to the Riccati equation (regardless of being positive-definite) are:
$$\frac 12P=\pmatrix{0&0 \\ 0&1},\; \pmatrix{k(k-1)^2& k(k-1)\\ k(k-1)&k},\; \pmatrix{k(k+1)^2&k(k+1)\\ k(k+1)&k+1}$$
Hence the closed-loop state matrix would be:
$$A_c=\pmatrix{k&1 \\ 0&-1},\quad \pmatrix{k& 1\\ 2k(1-k)&1-2k},\quad\pmatrix{k&1\\ -2k(k+1)&-2k-1}$$
You can see that the eigenvalues of the first matrix are $\{-1,k\}$, the second one $\{1,-k\}$ and for the third one $\{-1,-k\}$. So the second one is unstable and the stability conditions are $k<0$ for the first $P$ and $k>0$ for the third $P$.
In conclusion, the first $P$ is positive semidefinite and the third $P$ is positive definite for $k>0$. So the optimal policy depends on the poles of the open-loop system.
