# What is $\max\langle x,Ax\rangle$ over subspaces non-invariant under $A$?

Let $$A$$ be an Hermitian matrix in a vector space $$V$$, and let $$U\le V$$ be a subspace of $$V$$.

If $$U$$ is invariant under $$A$$, then the maximum of $$\langle x,Ax\rangle$$ over all unit vectors $$x\in U$$ equals the largest eigenvalue of $$A$$ whose eigenvector is in $$U$$ (and we know $$U$$ is spanned by eigenvectors of $$A$$, as otherwise it wouldn't be invariant under its action). Similar ideas are used for example to prove the min-max principle.

What about subspaces $$U$$ that are not invariant under $$A$$? More precisely, is there a way to find $$\max_{x\in U}\frac{\langle x,Ax\rangle}{\|x\|^2}$$ for arbitrary subspaces $$U$$? Of course, feel free to remove the Hermitianity constraint if the problem is better posed in a more general setting.

Decomposing an arbitrary $$x\in U$$ as $$x=\sum_k c_k x_k=\sum_j d_j u_j$$ where $$x_k$$ are a basis of eigenvectors of $$A$$ and $$u_k$$ an orthonormal basis for $$U$$, we have $$\langle x,Ax\rangle = \sum_k \lambda_k |c_k|^2,$$ but the problem is that the maximisation is constrained to those coefficients $$(c_k)$$ such that $$\sum_k c_k x_k\in U$$,

• If $P$ is the projector onto the subspace $U$ then for all $x\in U$ we have $\langle x, Ax\rangle = \langle P x, A P x \rangle = \langle x, PAP x\rangle$. Then $PAP$ Is a Hermitian matrix which leaves the subspace $U$ invariant and so we can solve it using the same method presented in the question. Would such a method be sufficient? Jun 1, 2020 at 21:25
• @Rammus , you may be waiting for someone to copy your comment and write it as an answer.
– user91684
Jun 1, 2020 at 22:06
• @Rammus that sounds like a good approach. Maybe complemented by an observation about the relation between the largest eigenvalues of $A$ and $PAP$
– glS
Jun 1, 2020 at 23:22
• @glS I'm not sure if I know of any relation that is not just trivial, i.e., $\|PAP\| \leq \|P\|\|A\|\|P\| = \|A\|$. This can be equality if the eigenvector corresponding to the largest absolute eigenvalue is contained within $U$. I'll update my answer if I think of something though. Jun 2, 2020 at 8:06
• @glS It's not quite equivalent to the unconstrained maximization of $PAP$. To see this imagine $P$ projects onto the eigenspace of the strictly negative eigenvalues of $A$ (assuming it has some). Then, maximizing over U would give something strictly negative whereas maximizing over $V$ would give $0$. If you add an absolute value around the inner product then these two optimizations would be the same. Jun 3, 2020 at 7:50

Let P be the projector onto the subspace $$U$$. Then for all $$x \in U$$ we have \begin{aligned} \langle x, A x \rangle &= \langle P x, A P x \rangle \\ &= \langle x, PAP x\rangle. \end{aligned} Therefore, we have $$\max_{x\in U}\frac{\langle x,Ax\rangle}{\|x\|^2} = \max_{x\in U}\frac{\langle x,PAPx\rangle}{\|x\|^2}.$$ Now $$PAP$$ is an operator that leaves the subspace $$U$$ invariant and so as noted in the question the maximum is given by the largest eigenvalue of $$PAP$$ whose eigenvector is in $$U$$. Note that we have an upper bound of $$\lambda_{\max}(A)$$ (largest eigenvalue of A) which can be seen be noticing the original problem is upper bounded by the same problem but with a maximization over the whole space $$V$$.