3
$\begingroup$

I have the standard stochastic, linear time varying system

$dx(t) = (A(t)x(t) + B(t)u(t))dt + G(t)dw(t) $ with $x(t_0) = x_0$

with quadratic cost

$J = x(t_F)^TQ_Fx(t_F) + \int_{t_0}^{t_F}\left( x(t)^TQ(t)x(t)+u(t)^TR(t)u(t)\right)dt$

Here $dw(t)$ is a white Gaussian random process with zero mean and covariance $W(t)dt$. I need to convert this to a discrete system. So I need the discrete version of $x(t_0), A(t), B(t), G(t), W(t), Q_F, Q(t),$ and $R(t)$.

This was my first guess, using the subscript $t$ to denote the discrete equivalent and $T$ is the discrete time step:

$A_t = e^{A(t)T}, \hspace{3 mm} B_t = \Big(\int_{\tau = 0}^Te^{A\tau}d\tau\Big) B(t), \hspace{3 mm} G_t = \Big(\int_{\tau = 0}^Te^{A\tau}d\tau\Big) G(t), \hspace{3 mm} W_t = A_t\Big(A_t^{-1}W_t \Big), $

$Q_{N_T} = Q_NT, \hspace{3 mm} Q_t = Q(t)T, \hspace{3 mm} R_t = R(t)T, \hspace{3 mm} x_{0_t} = x_{0}$.

I've search publications and text books, but I can't find anything that converts a stochastic continuous to stochastic discrete and considers both the $W$ and $G$ terms. I did find a paper which gave me the $W$ conversion above, under certain assumptions, but said nothing of the rest of the system. My questions:

1) Is this correct?

2) If not, what is correct?

3) If this cannot be done for time-varying systems, what would be the equivalent for an LTI system?

4) Given a both the continuous version and discrete version, if I controlled both systems with a continuous and discrete LQR/LQG feedback, respectively, will I get the same results?

Edit: Dimensions Clarification

Here $x(t)\in\mathbb{R}^{n\times 1}$ is the state, $u(t)\in\mathbb{R}^{m\times 1}$ is the control, and $dw(t)\in\mathbb{R}^{p\times 1}$ is a sequence of white Gaussian random noises with zero mean and covariance matrix $ W(t)\in\mathbb{R}^{p \times p}$. Also, we have the state matrix $ A(t) \in\mathbb{R}^{n\times n}$, the control matrix $B(t)\in\mathbb{R}^{n\times m}$, and $G(t)\in\mathbb{R}^{n\times p}$.

Problem 1: In the Wikipedia article mentioned in the comments, the system is time invariant, whereas my system is time varying. Additionally, they set $p=n$ and $G(t) =$ an $n\times n$ identity matrix $I_{n\times n}$. Also $W\in \mathbb{R}^{n\times n}$. Thus, the discretization of the covariance matrix, $W_t = A_t\Big(A_t^{-1}W_t \Big)$, gives an $n \times n$ matrix, which is incorrect in my case, as it should be $p\times p$.

I assume there should see the term $GWG^T$ somewhere, as $dw(t)$ has covariance $W$ so $Gdw(t)$ should have covariance $GWG^T$.

Problem 2: The article says it uses a zero-order hold for the stochastic system conversion, but in the derivation they use a non-stochastic system. You can see that in the stochastic case, for both continuous and discrete $G = I_{n\times n}$, i.e., the weights don't change during the conversion. I fail to see why this is true.

Problem 3: In the discretization, we have

$\int_0^{kT}e^{A(kT-\tau)}Bu(\tau)d\tau$

and assume that $u$ is constant over this interval, which gives us the well known result for system converion. When considering the $G$ term, I assume we get something similar:

$\int_0^{kT}e^{A(kT-\tau)}Gdw(t)d\tau$

Does this integral even mean anything, technically speaking? Most of the literature seems to brush off the stochastic integral equation. Also, are we assuming the noise as constant over this interval in the same way we assume the control is constant over the interval?

$\endgroup$
4
  • $\begingroup$ You are on the right way, the transformation is straightforward, I don't understand why you cannot find this, I remember this when I studied this topic. Look for the papers and book of Bertsekas and Shreve. $\endgroup$
    – Alexander Vigodner
    Aug 15, 2014 at 20:07
  • $\begingroup$ I'm sorry but I don't see how this is straightforward. I understand the transformation in the non-stochastic case but the $G(t)dw(t)$ term causes some problems. Also, converting the $W$ term isn't very straightforward (especially considering I've found publications on this alone). I agree though, I don't understand why I can't find this either! It seems like it should be readily available in a standard text book. But I've looked in Bertsekas and Shreve, and they don't cover this (at least, I did not see this). $\endgroup$
    – Sal
    Aug 15, 2014 at 20:45
  • $\begingroup$ That what I found immediately en.wikipedia.org/wiki/Discretization. It has references, so you can go deeper. $\endgroup$
    – Alexander Vigodner
    Aug 15, 2014 at 21:40
  • $\begingroup$ I addressed the Wikipedia article you linked in my original post. Also, I'd like to stress that my system is time varying and not time invariant, like the system in the Wikipedia article (and in most of the literature). $\endgroup$
    – Sal
    Aug 16, 2014 at 2:08

1 Answer 1

Reset to default
2
$\begingroup$

First of all $e^{A(t)}$ doesn't mean anything for time varying systems. You need to use state transition matrix $\Phi(t, t_0)$ as in the following:

$$A_t = \Phi(t + T, t)$$ $$B_t = \int_t^{t+T} \Phi(t+T, \tau) B(\tau) d \tau$$

assuming $u_t$ is constant over time $[t, t+T)$. But in this case I believe $G_t$ cannot be written separately. The system becomes

$$x_{t+1} = A_t x_t + B_t u_t + v_t$$

where

$$v_t := \int_t^{t+T} \Phi(t+T, \tau) G(\tau) w(\tau) d \tau$$

is a zero mean process with the covariance matrix

$$V_t := E[v_t v_t^T] = \int_t^{t+T} \Phi(t+T, \tau) G(\tau) W(\tau) G^T(\tau) \Phi^T(t+T, \tau) d \tau$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .