# Is it correct to say a state-space model has the Markov property?

in control theory, there exists a mathematical model of a control loop, called the state-space model. It allows for the computation of a state vector $$x$$ at discrete time $$k$$. For the computation, it requires knowledge of the previous state vector at $$k-1$$ only.

So, I wonder: Is it correct to say the state-space model has the Markov property? Markov property is also called memoryless property, but it is defined in the context of probability distributions. So I am not sure if my statement is valid.

• Well, you can say almost everything if you properly explain what you mean. :) But if you ask if this is a common saying in the control community, then I would rather say "no". You can speak about Markov jump systems, no problem, but typically we do not say "Markov property" for an arbitrary system. Jun 7, 2019 at 11:58
• thx, Arastas! So you say "Markov property" is not commonly said within this context. Is "memoryless property" used? Or neither? I mean, isn't this one of the crucial properties of a state-space model?
– Luk
Jun 8, 2019 at 6:00
• I can only say that I did not ever see someone using any special term to describe this property. Jun 8, 2019 at 20:40

Markov property is related to the probabilistic model of the state-space equations. $$\begin{array}{l} {{\bf{x}}_k} = {{\bf{f}}_{k - 1}}({{\bf{x}}_{k - 1}},{{\bf{u}}_k}) + {{\bf{v}}_k} \Leftrightarrow p({{\bf{x}}_k}|{{\bf{x}}_{k - 1}},{{\bf{u}}_k})\\ {{\bf{y}}_k} = {{\bf{h}}_k}({{\bf{x}}_k},{{\bf{u}}_k}) + {{\bf{w}}_k} \Leftrightarrow p({{\bf{y}}_k}|{{\bf{x}}_k},{{\bf{u}}_k}) \end{array}$$ where $$\bf{v}_k$$ , $$\bf{w}_k$$ are process and measurement noise, respectively. This state–space representation which incorporate random inputs or noise sources along with random initial conditions, is called Gauss–Markov model.

Under the first-order Markov assumption, conditional probability distribution of future states of the process (conditional on both past and present states) depends only upon the present state (2).

Since any pth-order Markov process can be transformed to a first-order process (1), (3), then we say that any state-space model has first-order Markov property.

A general state space model can be very complicated. Dynamical systems with hysteresis will definitely not have the Markov property.

If you can write down your system as

$$x_{k+1} = f(x_k,u_k)$$

then it will have the Markov property because only the current state and the current action/actuation are important for the next state.