Stability considerations for a special type of LTV systems I have been considering the stability of an (nonautonomous) LTV system $\dot{x} = A(t)x$ of the form
$\begin{bmatrix}\dot{x}_{1}\\ \dot{x}_{2} \end{bmatrix} = \begin{bmatrix}-a(t) & -b(t)\\ 1 & 0 \end{bmatrix} \begin{bmatrix}x_{1}\\x_{2}\end{bmatrix}$,
where we know that $a(t), b(t) > 0$ and both $a(t), b(t)$ are both continuous, differentiable, and bounded. By simulating such a system, it clearly seems to be stable, but I do not quite see how to prove that this is true and which restrictions that apply. By computing the eigenvalues we get
$\lambda_{1,2}(t) = \frac{-a(t) \pm \sqrt{a(t)^{2} - 4b(t)}}{2} < 0\; \forall\; t $,
with the restrictions we placed on $a(t), b(t)$. However, as emphasized by, e.g., Khalil (Nonlinear Systems (2002), Section 4.6), the eigenvalues of $A$ cannot be used to assert stability of such a system. If we instead fix $a, b$, then, by the same eigenvalues, the (autonomous) LTI system $\dot{x} = Ax$ is stable as long as $a, b > 0$.
I know that asserting the stability of general LTV systems is difficult, and from what I can see, most of the literature refer to numerical integration to find the transition matrix of the system. But is there really no other way to discuss the system shown above? The most "obvious" alternative that comes to mind is to find an appropriate Lyapunov function candidate, but I do not seem to be able to come up with a good candidate (?)
 A: Since you mentioned that you do not know beforehand the actual trajectories of $a$ and $b$, then no LTV method will be applicable.
A possible way is to treat those time-varying coefficients as time-varying uncertainties/parameters which both take values in some intervals; e.g. $a\in[a^-,a^+]$ and $b\in[b^-,b^+]$.
A first possible approach is to check whether the system is quadratically stable, that is, whether the function $V(x)=x^TPx$, $P=P^T\succ0$, is a Lyapunov function for the system. In the present case, this is equivalent to the feasibility of the following Linear Matrix Inequalities
$$A(a,b)^TP+PA(a,b)\prec0$$ for all $(a,b)\in\{a^-,a^+\}\times\{b^-,b^+\}$.
Depending on the bounds you may try to solve for that but such a matrix $P$ may not exist. The reasons are we are restricting ourselves to quadratic Lyapunov functions and that are allowed for a quite general classes of signals, including discontinuous ones.
The next step is to look at a robust stability criterion where you can try to find a Lyapunov function of the form $V(x)=x^TP(a,b)x$, $P(a,b)=P(a,b)^T\succ0$. This would lead to a stability condition of the form
$$\dot{a}\dfrac{\partial P}{\partial a}+\dot{b}\dfrac{\partial P}{\partial b}+A(a,b)^TP(a,b)+P(a,b)A(a,b)\prec 0$$
which must hold for all the values for $(a,b)$ and all values for $(\dot{a},\dot{b})$. This is much more complicated to consider and numerical methods are often used to check such conditions. The advantage of this approach is that we can capture information on the derivative of the parameters and it can be proven that if the system with frozen eigenvalues is stable, then it will remain stable provided that the rates of variation of the parameter is small enough.
In other words, the stability of the frozen LTV/LPV systems implies that of the LTV/LPV system provided that the rates of variation of the parameters are sufficiently small.
A: The time varying system you propose can be unstable depending the particular choice for $a(t)$ and $b(t)$. In order to make a counter example a little simpler to analyse I am ignoring that $a(t)$ and $b(t)$ both need to be continuous. Namely the following time varying system is unstable
$$
A(t) = \left\{
\begin{array}{l l}
A_1 & \text{if }\ (t\!\mod 5\pi/4) < \pi \\
A_2 & \text{otherwise}
\end{array}
\right. \tag{1}
$$
$$
A_1 =
\begin{bmatrix}
-0.2 & -0.25 \\ 1 & 0
\end{bmatrix}, \quad
A_2 =
\begin{bmatrix}
-0.8 & -4 \\ 1 & 0
\end{bmatrix}. \tag{2}
$$
In case my notation is unclear, $A(t)$ alternates between being equal to $A_1$ and $A_2$, where $A_1$ is active for $\pi$ time units and $A_2$ is active for $\pi/4$ time units.
When $A(t)$ is periodic with period $T$, such that $A(t+T) = A(t)\ \forall\ t \in \mathbb{R}$ then the stability can be determined by looking at the state transition matrix $\Phi(\tau+T,\tau)$ (with $\tau$ any constant real value). Namely, this state transition repeats every $T$ time units and thus one can do a discrete time stability analysis using the eigenvalues of $\Phi(\tau+T,\tau)$. For discrete time stability analysis of a linear time invariant system the absolute value of the eigenvalues needs lie withing the unit circle, instead of negative real part for the continuous time case.
When considering the example above, or any linear time varying system with piecewise constant dynamics, the state transition matrix from the start till the end of one active period of $A_i$ (with $i$ either $1$ or $2$) can be obtained using matrix exponentials
$$
\Phi(\tau_i+T_i,\tau_i) = e^{A_i\,T_i}, \tag{3}
$$
with $\tau_i$ the start of the active mode and $T_i$ the time length of the active mode (such that $T = T_1 + T_2$). A possible $\Phi(\tau+T,\tau)$ can be obtained using
$$
\Phi(\tau+T,\tau) = \Phi(\tau_1+T_1,\tau_1)\,\Phi(\tau_2+T_2,\tau_2) = e^{A_1\,T_1}\,e^{A_2\,T_2}. \tag{4}
$$
When substituting $(2)$ and the corresponding active periods $T_i$ into $(4)$ and evaluating the absolute value of the eigenvalues yields $|\lambda_1| \approx 0.13$ and $|\lambda_2| \approx 2.18$. It can be noted that $|\lambda_2| > 1$, so $\lambda_2$ lies outside of the unit circle and thus it can be concluded that a linear time varying system using $(1)$ would be unstable.

It is probably also possible to modify my counter example slightly to obtain a counter example which does have that $a(t)$ and $b(t)$ both are continuous. For example instead of instantaneously switching, having a quick linear interpolation between the two around the switching times. However, the evaluation of the state transition matrix then becomes a little more complicated.

The best tool for analyzing the stability of a linear time varying system in general would probably be Lyapunov functions. For example by looking for a, potentially time varying, common quadratic Lyapunov function
$$
V(x) = x^\top P(t)\,x, \tag{5}
$$
with $\alpha_1 \prec P(t) \prec \alpha_2\ \forall\ t$, where $0 < \alpha_1 \leq \alpha_2$ and $\alpha_2$ bounded, and the time derivative of $(7)$ should satisfy
$$
\dot{V}(x) = x^\top \left(A^\top\!(t) P(t) + \dot{P}(t) + P(t)\,A(t)\right) x \leq 0 \tag{6}
$$
and thus $A^\top\!(t) P(t) + \dot{P}(t) + P(t)\,A(t)$ should be a negative definite matrix for all $t$.
For the general time varying case I am not aware of structured ways of looking for such $P(t)$. However, when limiting $P(t)$ to be constant one can look for a solution using linear matrix inequalities. Namely, the previous requirements are equivalent to the following inequalities
\begin{align}
P &\succ 0, \\
A^\top\!(t)\,P + P\,A(t) &\preceq 0,\ \forall\ t.
\end{align}
One way to look for such a $P$ could be taking a lot of samples of $A(t)$ using different $t$ values (or as KBS mentioned, only the vertex points obtained from the extremas of $a(t)$ and $b(t)$). Use those evaluated values of $A(t)$ to construct a set of linear matrix inequalities, which can be attempted to be solved using a computer.
I suspect that not every stable linear time varying system can be proven to be stable using a constant common quadratic Lyapunov function. More relevant analysis tools can also be found in the sub-field of hybrid system, using topics like minimum dwell time.
