Linear ODE systems $x'=Ax$ are well understood. Suppose I have a quadratic ODE system where each component satisfies $x_i'=x^T A_i x$ for given matrix $A_i$. What resources, textbooks or papers, are there that study these systems thoroughly? My guess is that they aren't completely understood, but it would be good to know more about what has been done.
You are right, these systems are not completely understood even in the simple case of dimension $n=2$ (i.e., on the plane). You can check out this book on planar quadratic differential equations. In general you can consider systems of the form $$ \dot x_i=x_i(c_i+(Ax)_i), $$ which is slightly less general than you are asking for (the notation $(Ax)_i$ means the $i$th element of the vector $Ax$). These are the so-called Lotka--Volterra systems in mathematical ecology. There is an equivalent system defined on the simplex $$ \dot p_i=p_i((Ap)_i-p\cdot Ap), $$ which is called the replicator equation. There are a lot of open questions about these equations. A good book to look for some existing theory is Evolutionary games and popualtion dynamics (or just google the names).