1
$\begingroup$

I recently went through Kalman's paper "When is a Linear Control System Optimal" published in 1964. The paper makes me wonder whether the following statement is true: Any stabilizing control law is optimal for some LQR problems.

For the linear system $$ x_{t+1} = Ax_t + Bu_t, $$ and the average of a quadratic cost function \begin{equation} J(\pi) = \lim_{T\rightarrow\infty} \sum_{t=0}^{T-1}\left(x_t'Qx_t + u_t'Ru_t \right), \end{equation} where $Q$ is positive semi-definite and $R$ is positive definite, $(A,B)$ is controllable, and $(Q^{1/2},A)$ is observable, we know the policy $K$ such that $\pi(x) = K x$ is optimal for the cost $J(\pi)$ when $$ K = -(R+B'PB)^{-1}B'PA, \tag{a} $$ where $P$ is the unique positive definite solution of the algebraic Riccati equation $$ P = Q + A'PA- A'PB(R+B'PB)^{-1}B'PA \tag{b}. $$ We say $K$ is optimal for the LQR problem defined by $Q\succeq 0$ and $R\succ 0$ if conditions (a) and (b) are satisfied.

Or equivalently, we can say $K$ is optimal for $Q$ and $R$ if $$ \begin{aligned} Q + A'P(A+BK) - P =0,\\ RK+ B'P(A+BK)=0,\\ P\succeq0,\ \ Q\succeq 0, R\succ0, \end{aligned} \tag{c} $$ are satisfied.

The statement is that for any stabilizing $K$, there always exists some $Q\succeq 0$ and $R\succ 0$ such that conditions (a) and (b) or conditions (c) are satisfied (Any stabilizing control law is optimal for some LQR problems.)

I want to show that the statement is true if $B\in\mathbb{R}^{n\times m}$ with $m<n$ has rank $m$.

My effort:

I tried to show the statement is false. I select some stabilizing $K$. But for every $K$ I tried, I can always find $Q$ and $R$ that make $K$ optimal.

A similar statement can be found in Theorem 7 of Kalman's paper. But in the original paper, Kalman adopted a different form of the cost function (i.e., the control is scalar and an additional cost term $x_t'r u_t$ is considered).

$\endgroup$
4
  • 2
    $\begingroup$ I believe your question is equivalent to this question. $\endgroup$ Feb 19, 2022 at 6:01
  • 1
    $\begingroup$ The discrete-time has been considered more recently and the results and conclusions may differ from those in the continuous-time case. You may look at some of the papers there: scholar.google.com/…. $\endgroup$
    – KBS
    Feb 19, 2022 at 12:43
  • $\begingroup$ @KwinvanderVeen Thanks for the reference. From their post, the conclusion is every $K$ (not necessarily stabilizing) is optimal to a quadratic performance index that includes a cross-product term between the state and control. We don't have the cross-product term. Then several sufficient and necessary conditions need to be satisfied for a stabilizing $K$ to be optimal for some $Q$. These conditions are known in Theorem 1 of ieeexplore.ieee.org/abstract/document/6880317 and the discussion after that. $\endgroup$ Feb 19, 2022 at 22:12
  • $\begingroup$ @KBS thanks for sharing. I checked Prof. HU's papers before I posted the question. No answer to this qustion was spotted from his recent papers about inverse optimal control. $\endgroup$ Feb 19, 2022 at 22:41

1 Answer 1

1
$\begingroup$

The statement that any stabilizing control law is optimal for some LQR problems is not necessarily true even when $B$ has rank $m$. The necessary and sufficient conditions for a stabilizing $K$ to be optimal can be found in Theorem 4.1 and Theorem 5.1 of [R1].

[R1] also gives an example in which $K$ is stabilizing under $(A,B)$ but not optimal for any given $Q\succeq 0$ (see Section 6 of [R1]).

[R1] SUGIMOTO, KENJI, and YUTAKA YAMAMOTO. "Solution to the inverse regulator problem for discrete-time systems." International Journal of Control 48.3 (1988): 1285-1300.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .