2
$\begingroup$

Consider the usual linear process $x_{t+1} =Ax_t+Bu_t+Cw_{t+1}$ where $w_t$ is an independent and identically distributed $N(0,I)$ process. The objective is $$ V=E\sum_{t=0}^\infty \beta^t(x_t'Qx_t + u_t'Ru_t + x_t' S u_t) $$ The problem is identical to a stochastic LQR, but with the additional state-control interaction term $x_t' S u_t$. Is there a way to reduce this problem to the usual LQR formulation? Alternatively, is there a reference treating this general case?

$\endgroup$
2
$\begingroup$

Remark: you want to put a 2 your $x_t^\top S u_t$ term to save some headaches later, and throughout my hints I assume there is 2 on the cross term. I will also neglect the discounting factor $\beta$.

This is known as LQR with a cross term (or cross coupled LQR). There are a few ways to approach solving the problem: working out the solution through Hamilton-Jacobi-Bellman, forming the Hamiltonian, or "guessing" at the solution. For the sake of simplicity, I will go with the last one.

Start by defining $$ u_1=u_t+R^{-1} S^\top x_t, $$ then define your dynamics and cost in terms of $u_1$. This will yield a new $A$ matrix: $$\bar{A}=A-BR^{-1}S^\top,$$ and a new $Q$ matrix: $$\bar{Q}=Q-SR^{-1}S^\top.$$ This provides us with a new condition that needs to be satisfied: $(A-BR^{-1}S^\top,Q-SR^{-1}S^\top)$ is detectable.

Now with $\bar{A}$, $\bar{Q}$, and the resulting modified cost for $u_1$, we get back to a known LQR without cross terms and $$ u_t=-\underbrace{R^{-1}(S^\top+B^\top P)}_{K}x_t. $$

As for a reference, I have to imagine it is hidden away in one of Bertsakas' books or Kumar and Varaiya.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.