Why an LTI system with some zero eigenvalues still stable? The textbook says an LTI system $\dot x=Ax$ is stable if and only if the eigenvalues of $A$ have the strictly negative real part. However, I found a counterexample. If
$$A= \begin{bmatrix}-3 &  -1 &  -1\\
     1 &  -0.5 &  -0.5\\
     1 &  -0.5  & -0.5\end{bmatrix}$$
The state response of this system is convergent, and $x_2 = x_3$. The system is stable even if an eigenvalue of $A$ is $0$. Am I wrong?
 A: Unfortunately, most of the comments are incomplete or inaccurate, so I am going to post a full answer here.
Let us start with definitions and a dynamical system $\dot{x}(t)=f(x(t))$, $x(0)=x_0$, where $f$ is such that there exist a solution to that dynamical system and define $x(x_0,t)$ to be that solution.
Assume further that $x^*$ is an equilibrium point for that system; i.e. $f(x^*)=0$. Then, this equilibrium is

*

*stable if for all $\epsilon>0$, there exists an $\eta=\eta(\epsilon)>0$ such that $||x(0)-x^*||\le \eta$ implies that $||x(x_0,t)-x^*||\le\epsilon$ for all $t\ge0$.

*attractive if there exists an $\eta>0$ such that $||x(0)-x^*||\le \eta$ implies that $||x(x_0,t)-x^*||\to0$ as $t\to\infty$.

*asymptotically stable if it is both stable and attractive.

*unstable if it is not stable.

When those properties holds for all $x(0)$, then the equilibrium is said to be globally stable, globally attractive and globally asymptotically stable. It is also important to stress that stability properties are properties of equilibrium points and not of systems, unless in the global case where those properties are transferred to the system because of the global nature of the properties. I will come back to that later.
So, what does that mean?

*

*Stability means that if we start close enough from an equilibrium point, we remain in a neighborhood of that equilibrium point.

*Attractivity means that if we start close enough from an equilibrium point, we converge to that point.

*Asymptotic stability means that if we start close enough from an equilibrium point, we remain in a neighborhood of that equilibrium point and we ultimately converge to it.

So, we may be tempted to say that attractivity implies asymptotic stability but we can find systems for which an equilibrium point which are attractive but not asymptotically stable. This is the case of the system
$$\begin{array}{rcl}
\dot{x}_1&=&x_1^2-x_2^2\\
\dot{x}_2&=&2x_1x_2
\end{array}$$
for which the equilibrium point $x^*=0$ is attractive but not asymptotically stable.
Alright that was for nonlinear systems, but what about linear systems?
For linear systems of the form $\dot{x}(t)=Ax(t)$, $x(0)=x_0$, we are mostly interested in the zero solution of the system (but not always) and global stability properties of the zero equilibrium point. Those stability properties can be linked to the eigenstructure of the matrix $A$ and in fact we have that

*

*The zero equilibrium point is stable if and only if the spectrum of the matrix $A$ is included in the closed left-half plane and the eigenvalues on the imaginary axis are semisimple (i.e. their algebraic and geometric multiplicity are the same.

*The zero equilibrium point is asymptotically stable if and only if the spectrum of the matrix $A$ is included in the open left-half plane; i.e. all the eigenvalues have strictly negative real part.

An nice way of reformulating those statements is in terms of the properties of the impulse response of the system where we assume that the system is both observable and controllable (observability can be relaxed by looking at the response of all states, but controllability is necessary):

*

*The zero equilibrium point is stable if and only if the impulse response is bounded.

*The zero equilibrium point is asymptotically stable if and only if the impulse response converges to zero.

This one of the reasons we say the system is stable or unstable, it is because the system uniquely defines the property of the equilibrium point, which is unique in that case.
Let us take some examples and especially show the importance of the multiplicity of the eigenvalues on the imaginary axis.

*

*The system $\dot{x}=u$ as one simple eigenvalue at zero and is therefore stable. Its impulse response of the transfer function $G(s)=1/s$ is $H(t)$, where $H$ is the Heaviside step function which is bounded.

*The system $\ddot{x}=u$ has two eigenvalues at zero which is not semisimple as the geometric multiplicity is one (Jordan form). It is, therefore, not stable. Picking $y=x$, the transfer function is $G(s)=1/s^2$ and its impulse response $tH(t)$, which is growing unbounded with time.

In your example, the eigenvalues are $-2-i,-2+i,0$, so the system is globally stable and the trajectories will remain bounded. However, if
$$x(0)\in\mathrm{span}\left(\begin{bmatrix}1\\ 0\\ 0\end{bmatrix},\begin{bmatrix}0\\ 1\\ 1\end{bmatrix}\right),$$
then we will have that $x(t)\to0$ as $t\to\infty$. The reason is that if we start from a subspace that is orthogonal to the subspace associated with the zero eigenvalues, the trajectory will remain in the stable subspace associated with the eigenvalues with negative real part and will, therefore, converge to zero.
