# Converse linear quadratic optimal control

It is well known that for a linear time invariant system

$$\dot{x} = A x + B u \tag{1}$$

with $$(A, B)$$ controllable, there exists a static state feedback $$u = -K x$$ such that the cost function

$$J = \int_0^{\infty} x^T Q x + u^T R u \, dt \tag{2}$$

is minimized, assuming $$Q \geq 0$$ (positive semi-definite) and $$R > 0$$ (positive definite). The gain $$K$$ is the solution of the algebraic Riccati equation:

\begin{align} 0 &= A^T P + P A - P B R^{-1} B^T P + Q \\ K &= R^{-1} B^T P \\ P &= P^T \geq 0 \end{align}

known as linear quadratic regulator (LQR). However, I wonder whether the converse also holds?

That is, given a stabilizing $$K_s$$ (such that $$A - B K_s$$ is Hurwitz), do there exist matrices $$Q \geq 0$$ and $$R > 0$$ such that $$u = -K_s x$$ minimizes $$(2)$$ given $$(1)$$? Or put differently:

Question: Is every stabilizing linear state feedback optimal in some sense?

• Can you pick $$J=\int_0^\infty\|K_sx+u\|^2\,dt?$$ – A.Γ. Apr 22 at 10:03
• @A.Γ. I guess so. I assume that $|| . ||^2$ means Euclidean norm squared. Then that would be the same as $Q = K_s^T K_s \geq 0$, $R = I > 0$ (identity matrix) and $N = K_s^T$ for the cost function including the mixed term $J = \int_0^{\infty} x^T Q x + u^T R u + 2 x^T N u \, dt$ which is also a standard LQR problem. However, how do you know that this will be the cost function which will give $K_s$ when solved? – SampleTime Apr 22 at 11:07
• @A.Γ. Ok, just noticed that $J = 0$ for $u = -K_s x$ so it is obviously a minimizer. That is pretty cool, if you want to post it as an answer, I will accept it. However, this surprises me as it means that every stabilizing state feedback is optimal in the LQR sense for some $Q$, $R$, $N$. I didn't expect that to be true in general... – SampleTime Apr 22 at 11:48
• @A.Γ. but this will also hold for an unstable state feedback. – Kwin van der Veen Apr 22 at 13:33
• @KwinvanderVeen I do not understand what you mean by "this". An "unstable" feedback cannot be a solution to LQR in the class of stabilizing controllers, hence, cannot be optimal unless you optimize over all linear controllers (which is not usually the case). – A.Γ. Apr 22 at 14:01

See the paper: Kalman, R. E. (1964). When is a linear control system optimal?. Journal of Basic Engineering, 86(1), 51-60.

The answer is positive at least for a class of systems. As far as I remember, the answer is also positive for a general LTI system, but I cannot find a reference at the moment.

UPDATE: Every linear system with nondynamic feedback is optimal with respect to a quadratic performance index that includes a cross-product term between the state and control, see [R1].

If you do not allow for the cross-product term, then several sufficient and necessary conditions are known, see for example [R2] and the references there.

[R1] Kreindler, E., & Jameson, A. (1972). Optimality of linear control systems. IEEE Transactions on Automatic Control, 17(3), 349-351.

[R2] Priess, M. C., Conway, R., Choi, J., Popovich, J. M., & Radcliffe, C. (2015). Solutions to the inverse lqr problem with application to biological systems analysis. IEEE Transactions on control systems technology, 23(2), 770-777.

• @SampleTime For that proposed solution you need to solve $A^\top P + P A - (P B + N) R^{-1} (B^\top P + N^\top) + Q = 0$, but $N\,R^{-1} N^\top = Q$, so $P=0$ is always a solution. Normally numerical LQR solvers try to find a "sensible" solution with $P=P^\top>0$, so will probably return an error when trying to solve the CARE with those $Q$, $R$ and $N$. Also that proposed cost function also returns zero when the state feedback is unstable. – Kwin van der Veen Apr 22 at 13:53
• @SampleTime Check also this one: Priess, M. C., Conway, R., Choi, J., Popovich, J. M., & Radcliffe, C. (2015). Solutions to the inverse lqr problem with application to biological systems analysis. IEEE Transactions on control systems technology, 23(2), 770-777. – Arastas Apr 23 at 11:08
• Please see the updated answer. – Arastas Apr 23 at 11:39
• @SampleTime I just checked, the guaranteed gain and phase margins for LQR only hold when $N=0$. For the proposed LQR problem with $N = K^\top$ the lower bound for these margins are actually zero. – Kwin van der Veen Apr 23 at 12:53
• @SampleTime I derived that myself. Namely in the single input case (so $R$ scalar) with $Q$ invertible it can be shown that the open loop in the Nyquist plot stays outside the circle around the minus point with radius $1-N^\top Q^{-1} N /R$. – Kwin van der Veen Apr 24 at 14:45

The problem is called the Inverse Problem of Optimal Control (see page 147 - 148).

Given a system $$\dot{x}=Ax+Bu,\qquad x(t_0)=x_0$$ $$z = \begin{bmatrix} Q^{1/2}& 0 \\ 0 & R^{1/2}\\ \end{bmatrix}\begin{bmatrix}x\\u \end{bmatrix},$$

with $$(A,B)$$ is stabilizable, $$(Q,A)$$ is detectable and $$R>0$$ (positive definite). The linear quadratic regulator problem is given by minimizing

$$\int_{0}^{\infty}z^Tz dt.$$

From Boyd et al 1994: The inverse problem of optimal control is the following. Given a matrix $$K$$, determine if there exists $$Q ≥ 0$$ and $$R > 0$$, such that $$(Q, A)$$ is detectable and $$u = Kx$$ is the optimal control for the corresponding LQR problem. Equivalently, we seek $$R > 0$$ and $$Q ≥ 0$$ such that there exists P nonnegative and P1 positive-definite satisfying

$$(A + BK)^T P + P(A + BK) + K^TRK + Q = 0, \quad B^T P + RK = 0$$

and $$A^T P_1 + P_1A < Q$$. This is an LMIP in $$P$$, $$P_1$$, $$R$$ and $$Q$$. (The condition involving $$P_1$$ is equivalent to $$(Q, A)$$ being detectable.)

• Thanks for the additional reference as well. From there it becomes clear that $P = 0$ is allowed because stability is guaranteed by $P_1 > 0$. – SampleTime Apr 23 at 17:49