It is well known that for a linear time invariant system

$$ \dot{x} = A x + B u \tag{1} $$

with $(A, B)$ controllable, there exists a static state feedback $u = -K x$ such that the cost function

$$ J = \int_0^{\infty} x^T Q x + u^T R u \, dt \tag{2} $$

is minimized, assuming $Q \geq 0$ (positive semi-definite) and $R > 0$ (positive definite). The gain $K$ is the solution of the algebraic Riccati equation:

$$ \begin{align} 0 &= A^T P + P A - P B R^{-1} B^T P + Q \\ K &= R^{-1} B^T P \\ P &= P^T \geq 0 \end{align} $$

known as linear quadratic regulator (LQR). However, I wonder whether the converse also holds?

That is, given a stabilizing $K_s$ (such that $A - B K_s$ is Hurwitz), do there exist matrices $Q \geq 0$ and $R > 0$ such that $u = -K_s x$ minimizes $(2)$ given $(1)$? Or put differently:

Question: Is every stabilizing linear state feedback optimal in some sense?

  • 2
    $\begingroup$ Can you pick $$J=\int_0^\infty\|K_sx+u\|^2\,dt?$$ $\endgroup$ – A.Γ. Apr 22 '19 at 10:03
  • $\begingroup$ @A.Γ. I guess so. I assume that $|| . ||^2$ means Euclidean norm squared. Then that would be the same as $Q = K_s^T K_s \geq 0$, $R = I > 0$ (identity matrix) and $N = K_s^T$ for the cost function including the mixed term $J = \int_0^{\infty} x^T Q x + u^T R u + 2 x^T N u \, dt$ which is also a standard LQR problem. However, how do you know that this will be the cost function which will give $K_s$ when solved? $\endgroup$ – SampleTime Apr 22 '19 at 11:07
  • $\begingroup$ @A.Γ. Ok, just noticed that $J = 0$ for $u = -K_s x$ so it is obviously a minimizer. That is pretty cool, if you want to post it as an answer, I will accept it. However, this surprises me as it means that every stabilizing state feedback is optimal in the LQR sense for some $Q$, $R$, $N$. I didn't expect that to be true in general... $\endgroup$ – SampleTime Apr 22 '19 at 11:48
  • 1
    $\begingroup$ @A.Γ. but this will also hold for an unstable state feedback. $\endgroup$ – Kwin van der Veen Apr 22 '19 at 13:33
  • $\begingroup$ @KwinvanderVeen I do not understand what you mean by "this". An "unstable" feedback cannot be a solution to LQR in the class of stabilizing controllers, hence, cannot be optimal unless you optimize over all linear controllers (which is not usually the case). $\endgroup$ – A.Γ. Apr 22 '19 at 14:01

See the paper: Kalman, R. E. (1964). When is a linear control system optimal?. Journal of Basic Engineering, 86(1), 51-60.

The answer is positive at least for a class of systems. As far as I remember, the answer is also positive for a general LTI system, but I cannot find a reference at the moment.

UPDATE: Every linear system with nondynamic feedback is optimal with respect to a quadratic performance index that includes a cross-product term between the state and control, see [R1].

If you do not allow for the cross-product term, then several sufficient and necessary conditions are known, see for example [R2] and the references there.

[R1] Kreindler, E., & Jameson, A. (1972). Optimality of linear control systems. IEEE Transactions on Automatic Control, 17(3), 349-351.

[R2] Priess, M. C., Conway, R., Choi, J., Popovich, J. M., & Radcliffe, C. (2015). Solutions to the inverse lqr problem with application to biological systems analysis. IEEE Transactions on control systems technology, 23(2), 770-777.

  • 1
    $\begingroup$ @SampleTime For that proposed solution you need to solve $A^\top P + P A - (P B + N) R^{-1} (B^\top P + N^\top) + Q = 0$, but $N\,R^{-1} N^\top = Q$, so $P=0$ is always a solution. Normally numerical LQR solvers try to find a "sensible" solution with $P=P^\top>0$, so will probably return an error when trying to solve the CARE with those $Q$, $R$ and $N$. Also that proposed cost function also returns zero when the state feedback is unstable. $\endgroup$ – Kwin van der Veen Apr 22 '19 at 13:53
  • 1
    $\begingroup$ @SampleTime Check also this one: Priess, M. C., Conway, R., Choi, J., Popovich, J. M., & Radcliffe, C. (2015). Solutions to the inverse lqr problem with application to biological systems analysis. IEEE Transactions on control systems technology, 23(2), 770-777. $\endgroup$ – Arastas Apr 23 '19 at 11:08
  • 1
    $\begingroup$ Please see the updated answer. $\endgroup$ – Arastas Apr 23 '19 at 11:39
  • 1
    $\begingroup$ @SampleTime I just checked, the guaranteed gain and phase margins for LQR only hold when $N=0$. For the proposed LQR problem with $N = K^\top$ the lower bound for these margins are actually zero. $\endgroup$ – Kwin van der Veen Apr 23 '19 at 12:53
  • 1
    $\begingroup$ @SampleTime I derived that myself. Namely in the single input case (so $R$ scalar) with $Q$ invertible it can be shown that the open loop in the Nyquist plot stays outside the circle around the minus point with radius $1-N^\top Q^{-1} N /R$. $\endgroup$ – Kwin van der Veen Apr 24 '19 at 14:45

The problem is called the Inverse Problem of Optimal Control (see page 147 - 148).

Given a system $$\dot{x}=Ax+Bu,\qquad x(t_0)=x_0$$ $$ z = \begin{bmatrix} Q^{1/2}& 0 \\ 0 & R^{1/2}\\ \end{bmatrix}\begin{bmatrix}x\\u \end{bmatrix},$$

with $(A,B)$ is stabilizable, $(Q,A)$ is detectable and $R>0$ (positive definite). The linear quadratic regulator problem is given by minimizing

$$\int_{0}^{\infty}z^Tz dt.$$

From Boyd et al 1994: The inverse problem of optimal control is the following. Given a matrix $K$, determine if there exists $Q ≥ 0$ and $R > 0$, such that $(Q, A)$ is detectable and $u = Kx$ is the optimal control for the corresponding LQR problem. Equivalently, we seek $R > 0$ and $Q ≥ 0$ such that there exists P nonnegative and P1 positive-definite satisfying

$$(A + BK)^T P + P(A + BK) + K^TRK + Q = 0, \quad B^T P + RK = 0$$

and $A^T P_1 + P_1A < Q$. This is an LMIP in $P$, $P_1$, $R$ and $Q$. (The condition involving $P_1$ is equivalent to $(Q, A)$ being detectable.)

  • $\begingroup$ Thanks for the additional reference as well. From there it becomes clear that $P = 0$ is allowed because stability is guaranteed by $P_1 > 0$. $\endgroup$ – SampleTime Apr 23 '19 at 17:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.