# Eigenvalues of real diagonal matrix times orthogonal projection

In the analysis of the stability of a dynamical system I came across the following Jacobi matrix $J$ (the eigenvalues of which determine stability): $$J=KP$$ where $K$ is an $m\times m$ real diagonal matrix and $$P=A\left(A^{T}A\right)^{-1}A^{T}$$ is an orthogonal projection. The non-square matrix $A$ is an $m\times n$ real matrix, $m\geq n$.

The eigenvalues of $P$ are 1 with algebraic multiplicity $n$ and 0 with algebraic multiplicity $m-n$ (see this proof exploiting the idempotence of $P$ and this proof arguging the multiplicity).

However, when you consider the whole matrix $J=KP$, I am not sure about its eigenvalues.

I am particularly interested if anything can be said about the sign of the the largest nonzero eigenvalue of $J$, as this is what matters for stability analysis.

Let $\lambda_{\max}(A)$ be the largest singular value of $A$; when $A$ is symmetric this is the largest absolute value of the eigenvalues of $A$.

Since $P$ is a projection, $\lambda_{max}(J)\le \lambda_{\max}(K)$. Equality can be attained: take $P$ to be the identity, or assume that the range of $P$ contains the largest eigenvector of $K$.

A similar inequality holds for the the maximal eigenvalue $\sigma_{\max}$, provided that $\sigma_{\max}(K)\ge0$. To see this, notice that any eigenvalue of $J=KP$ is also an eigenvalue of the symmetric matrix $PKP$. Since $P$ is a projection, $\|Pu\|\le\|u\|$ and therefore $$\sigma_{\max}(J) \le\sigma_{\max}(PKP) =\sup_{\|u\|\le1}\langle PKPu,u\rangle =\sup_{\|u\|\le1}\langle KPu,Pu\rangle \le\sup_{\|v\|\le1}\langle Kv,v\rangle =\sigma_{\max}(K).$$ For the eigenvalue statement between $KP$ and $PKP$, notice that if $KPv=\lambda v$, then $P^{1/2}KP^{1/2}(P^{1/2}v)=\lambda (P^{1/2}v)$. So all the eigenvalues of $KP$ are also eigenvalues of $P^{1/2}KP^{1/2}=PKP$, because $P$ is a projection. $A^{1/2}$ denotes the matrix square root of a nonnegative definite matrix $A$ (and $P$ is such, as a projection).

You can show that any \emph{nonzero} eigenvalue of $PKP$ is also an eigenvalue of $J$. It could be that $J$ is not diagonalisable, but this will happen only if there is a nonzero $v$ such that $Pv=v$ and $PKv=0$ yet $Kv\ne0$ ($v$ is the range of $P$ but $Kv$ is in the orthogonal complement of the range of $P$).

We will have equality in the second inequality if and only if the maximising $v$ (or \emph{a} such maximiser if there are many) is in the range of $P$. How far off we are depends on how close is the range of $P$ to a maximiser, and also on how close are the other eigenvalues of $K$ to the maximal one.

If $\sigma_{\max}(K)<0$, then taking $P=0$ shows that $\sigma_{\max}(J)$ can be larger than $\sigma_{\max}(K)$. But the same idea shows that $\sigma_{\min}(J)\ge \sigma_{\min}(K)$ if the latter is not positive.

So:

$K$ negative definite ---> $\sigma_{\max}(J)\le0$, and it will be zero unless $P$ is the identity.

$K$ not negative definite ---> $0\le\sigma_{\max}(J) \le \sigma_{\max}(K)$. Note that it doesn't mean that $K$ is nonnegative definite, all it means is that it has at least one nonnegative eigenvalue.

There analogous inequalities for $\sigma_{\min}$ also hold.

• Thank you, $\lambda_{max}\left(J\right)\leq\lambda_{max}\left(K\right)$ is a nice idea. However, this is only a statement about the absolute values of eigenvalues. This is not so helpful for my problem, as I care about the largest nonzero eigenvalue. A bound on that would be incredibly helpful. Unfortunately I have to live with the fact that $J$ is not symmetric, as it is dictated by the linear stability analysis of a dynamical system near an equilibrium point of interest. – Alexander Erlich Feb 14 '18 at 20:48
• Can I also ask for a reference on the inequality? If you have a good book recommendation that would help also. – Alexander Erlich Feb 15 '18 at 9:24
• Sorry, I don't really know books about this, but I am sure that more or less any linear algebra would have. You can have a look here en.wikipedia.org/wiki/Matrix_norm, the $\lambda_{\max}$ is the norm they call $\|A\|_{2,2}$, and they say it's sub-multiplicative: $\|KP\|\le \|K\|\|P\|$ and $\|P\|=1$ (or 0 if $P=0$). – YZS Feb 16 '18 at 12:00
• I had a mistake in my answer. $J$ is not symmetric but in many cases it will have eigenvectors, I will modify the answer. – YZS Feb 16 '18 at 12:24
• I was hoping you could please have a look at my comment/answer below. – Alexander Erlich Feb 19 '18 at 2:07

This is an extended comment on @YZS answer.

I appreciate that the largest singular values $\lambda_\text{max}$ satisfy $\lambda_{\text{max}}\left(KP\right)\leq\lambda_{\text{max}}\left(K\right)$. Let's say that we call $$\sigma_{\text{max}}$$ the largest non-zero eigenvalue. From running numerical tests, I strongly suspect that the eigenvalues are bounded in a very similar way to the singular values, perhaps even the same way, $\sigma_{\text{max}}\left(KP\right)\leq\sigma_{\text{max}}\left(K\right)$.

Consider this simulation result, which I found to be representative:

Here $\lambda_\text{max}$ are largest singular values and $\sigma_\text{max}$ largest non-zero eigenvalues. Every dot represents a randomly generated matrix $A$ of dimensions $6\times 10$, the entries of which were integers between $-100$ and $100$. For all simulations, $K$ was always the same diagonal matrix $$K=\text{diag}(-9,-9,-8,-8,-5,-4,-4,-1,1,3).$$ So clearly, $$\sigma_{\text{max}}\left(K\right)=3,\quad\lambda_{\text{max}}\left(K\right)=9.$$ I have repeated such simulations for various $K$, as well as different dimensions of $A$ (starting with a square and then increasing column size, growing $K$ accordingly). I have found that the stated inequalities for both singular and non-zero eigenvalues were always true in my simulations.

Of course such tests are not an exhaustive search, but can anyone prove or provide a plausibility argument why $\sigma_{\text{max}}\left(KP\right)\leq\sigma_{\text{max}}\left(K\right)$ might be correct?

Also, given that $KP$ is not symmetric, is it somehow obvious why I never find any imaginary eigenvalues in numerical simulations?

As an aside, as you change the dimensions of $A$ from a square to increasingly elongated rectangle, it appears that the largest nonzero eigenvalues $\sigma_\text{max}(K P)$ move further and further away from the (supposed) bound $\sigma_\text{max}(K)$. This trend also holds for the max. singular values. In the plot below, $\sigma_\text{max}(K)=-2$ and the dimension of $A$ changes from $6\times 6$ to $6 \times 40$, as the eigenvalues move further and further below the (supposed) bound $\sigma_\text{max}(K)$:

• Nice. You can show that the inequality holds for the largest eigenvalue as well. In fact $KP$ and $PKP$ have the same nonzero eigenvalues, and we have $\sigma_{\max}(K)=\sup_{\|v\|\le1} \langle Kv,v\rangle$. But $\sigma_{\max}(PKP)=\sup_{\|u\|\le1}\langle KPu,Pu\rangle$ is less than $\sigma_{\max}(K)$ because $\|Pu\|\le\|u\|$. I will modify my answer accordingly. – YZS Feb 20 '18 at 15:27
• Are you interested in $K$ negative definite ($\sigma_{\max}(K)<0$)? In that case, unless $P$ is the identity, the largest eigenvalue of $J$ is zero. I guess the largest nonzero eigenvalue of $J$ will have to be smaller (more negative) than $\sigma_{\max}(K)$, but I haven't tried out the details. – YZS Feb 20 '18 at 16:17
• Thank you for updating your reply! No, my $K$ are not necessarily negative definite, as in the case of the first numerical plot. – Alexander Erlich Feb 20 '18 at 16:35
• So in general the nonzero eigenvalues of $KP$ and $PKP$ are the same. Usually $KP$ will have $m$ eigenvalues that are real. Trouble can only occur if $K$ sends a vector in the range of $P$ to somewhere in the orthogonal complement: $K=\begin{pmatrix}1 & 0 \\0 & -1\end{pmatrix}$ and $P$ matrix of all $1/2$, then $KP$ is not diagonalisable. But if you take $K$ and/or $P$ to be random, then this situation is unlikely to occur. – YZS Feb 21 '18 at 14:05