What are the differences between mathematical systems theory, Dynamical Systems, and Optimization and Control and how are they related to each other?

One of the areas of research in Systems and Controls group at Georgia Institute of Technology is Mathematical systems theory. It seems that the electrical engineering department at Georgia Institute of Technology is the only electrical engineering department that studies Mathematical systems theory.

On the other hand, arxiv archive has categories like Dynamical Systems, Systems and Control and Optimization and Control; some of the authors of the papers in these categories are from electrical engineering departments.

What are the differences between mathematical systems theory, Dynamical Systems, Systems and Control, and Optimization and Control and how are they related to each other? what universities are well known for their research in these areas? are these applied math?

In general any system of the form: $$\dot x = f(x)$$ is a dynamical system. For example the simple harmonic oscillator is a dynamical system $\ddot x = - kx$. When the right hand side is linear we call it a linear system. When the right hand side takes the form, $$\dot x = f\left( {x,u} \right)$$ where $u$ is a input we can choose, we call it a controlled dynamical system (and here it ties in with control systems theory). Moreover when the right hand side is linear for the controlled dynamical system the theory we have is rich and encompasses many methods (from classical transfer functions to advanced state-space methods). When the right hand side has an explicit dependence on time $\dot x = f(x,t,u)$ we have non-autonomous systems.

When we search over the space of possible inputs $u$ under some cost functionals or optimality constraints, the succeeding theory is the one of optimal control which has ties with the calculus of variations and functional optimization.

So maybe one way to think about is that the first case, $\dot x = f (x)$ is at the most abstract level, the mathematicians domain whereas with some additional structure (maybe linearity for example) to the formulation; tools like transfer functions, Bode plots etc. becomes useful and then we kind of move over into the engineer's domain. This is most apt when talking about linear systems. With a non-linear dynamical system, the theory that is useful depends heavily on real and functional analysis (Lyapunov theory, zero dynamics, non-minimum phase systems, etc.), and is generally not approachable at the undergraduate level and will require some level of mathematical sophistication.

Indeed they are all related to each other because they are all their very heart dealing with deterministic differential equations of some form or the other. Mathematical systems theory tries to deal with dynamical systems at the most abstract level (with or without control) and this is related to bifurcation theory, chaos, etc. which as you pointed out is really the domain of applied mathematics. When we have an input output relation, the systems we consider generally has some real-world application so we move over into the engineering side and try to make it less abstract and more approachable.

I am not sure how to answer the question about research at universities because it depends on what kind of research. Are you talking about cutting edge controls and dynamics? Then you have Google, Boston Dynamics, MIT, ETH Zurich etc. working on very applied controls. Are you talking about systems theory research? Then that's much harder to answer, but there are applied mathematicians (often allied with the Electrical or Mechanical Engineering departments) all over the country chipping away at such research.

• Good answer! I'd add that people who research or apply control theory, by any of the names above, will gladly investigate stochastic as well as deterministic dynamical systems, whenever the resulting mathematics brings up interesting theoretical questions or solves relevant practical problems.
– Pait
Sep 8, 2016 at 10:51

The names of departments or posted research areas can be uninformative, and sometimes even misleading. So, your best bet is to focus on the subjectmatter.

Dynamical systems are mathematical models (of various phenomena) that consist of differential or difference equations. Such a system is described by, first, specifying the set of all possible states it can have, and then the set of rules how it goes from state to state. For example, an ordinary differential equation tells the instantaneous rate ${\bf v}({\bf x})$ at which the state of the system changes when the system is in state ${\bf x}$; i.e., ${d \over dt}{\bf x} = {\bf v}({\bf x})$. (See V. Arnol'd's Ordinary Differential Equations.)

When we want to use dynamical systems to model something that acts like a device; i.e., has an input signal fed into the system from the outside, and this signal affects how the system changes state, the above ODE is replaced by the control system: $${d \over dt}{\bf x} = {\bf v}({\bf x}, {\bf u}).$$ (The finite version of this is the Finite State Macnine, so I would recommend looking at these first, for a smoother initiation.) This need not always be a device: the system can be a biological organism, and the input signal can be a drug administered to it or the genetic mutation rate of the cells (e.g., Wan, F.M.Y., Sadovsky, A.V. and Komarova, N.L. (2010) Genetic Instability in Cancer: An Optimal Control Problem. Studies in Applied Mathematics 125 (1): 1 --38).

We are always concerned with the existence and uniqueness of ODE solutions, and, in the case of control systems, also with the sets of states reachable from a given state by applying the permissible controls ${\bf u}$. If every state trajectory incurs a cost, we are interested in optimal control (getting from one given state to another with minimal cost). A good introductory book on control systems in general is Control System Design by Freedland; on optimal control, see Bryson and Ho's Applied Optimal Control.

The mathematical apparatus used for the above (at least, for differential equations) is that of diffeomorphisms (notably, the inverse function theorem and the implicit function theorem, which are easy corollaries of each other) and differential geometry of tangent vector fields on manifolds. Two omnipresent parts of it are Lyapunov stability and Lie algebras. See Arnol'd's book I quoted above, as well as his Mathematical Methods of Classical Mechanics, for an elementary introduction. For a very broad overview, see Sussmann's survey paper DIFFERENTIAL-GEOMETRIC METHODS: A POWERFUL SET OF NEW TOOLS FOR OPTIMAL CONTROL.

Engineering design applications make also wide use of the Fourier and Laplace transforms: here the first basic concepts to learn are transfer function (also called frequency response) and resolvent (see Freedland's book).

• Another good answer! I'll add that most often in control theory and applications, a 2nd equation is necessary to describe the interaction between the object being modeled and the real world. The readout equation $y=h(x)$ described how the measured output $y$ depends on the state of the system, which by itself is usually unavailable and can only be accessed using physical devices called sensors. The issue of estimating $x$ from $y$ (or "observing" in the jargon) is as relevant, and sometimes more challenging, that that of controlling $x$ through $u$.
– Pait
Sep 8, 2016 at 10:56
• @Pait, thank you, and you make a very good point. State estimation is an area of research in its own right, so "relevant" is an understatement. I probably should have mentioned the Kalman filter in that context, but I hope the Asker just sees these comments.
– avs
Sep 8, 2016 at 16:58

Mathematical systems theory also studies dynamic systems and control of dynamic systems, but from an abstract algebraic perspective. Systems theory starts with system theoretic descriptions that are based on vector spaces. Mathematical systems theory does this as well, but then extends the ideas to systems theory over modules, e.g., matrices over arbitrary rings instead of the usual field R. In this perspective, the usual system theoretic notions take on new interpretation. For example, the rank of the controllability matrix condition: rank([B, AB, ... A^(n-1)B]) takes on the interpretation that it is the condition for generating a module over polynomials in the matrix A, with B being the generator.