# generalized Rayleigh Quotient

If $M$ is positive definite, $H$ is self-adjoint. Now consider the minimization problem:$$\min_{x\neq 0}\frac{(x,Hx)}{(x,Mx)}.$$ Note that this functional is homogeneous of degree 0. So we can just search the minimum on the unite sphere. And because of the continuity of the functional and that the unit sphere of a finite dimensional space is compact. We can find a solution $f$, and by further calculation(variational method), we can show that $f$ satisfies:$$Hf=bMf,\qquad \hbox{where b=\frac{(f,Hf)}{(f,Mf)}}.$$ Now here's my question. Continue to search the minimum in the subspace $\{y:(y,Mf)=0\}$. By the same continuity and compactness argument, the solution exists. Denote it $g$, i.e.$$\min_{y\neq 0, \ (y,Mf)=0}\frac{(y,Hy)}{(y,My)}=\frac{(g,Hg)}{(g,Mg)}$$ How can I show that $g$ satisfies $$Hg=cMg\qquad \hbox{where c=\frac{(g,Hg)}{(g,Mg)}}?$$

It's probably quite simple... I might think it to be too complicated. Anyone who can give me a hint?

I've just realized that one can always change this kind of generalized Rayleigh Quotient back to the standard Rayleigh Quotient. By changing back and using what has already been established about the relation between minimizing Rayleigh Quotient and the eigenvector of a Hermitian (if in real case, symmetric), I can derive the required equality. But it seems I'm taking a detour. Is there any way to make it more straightforward?

-
what's the domain and range of your operator $M$ and $H$? –  Jack Mar 17 '12 at 4:21
@Jack From an N-dimensional space $X$ to itself. –  henryforever14 Mar 17 '12 at 4:22

The argument goes using the same variational method modified with respect to the set of admissible variations. Denote $$A = \left\{y:\left(y,Mf\right)=0 \right\}$$ Now for an admissible variation $y\in A$ the statement that $g$ gives the ratio a minimum is expressed as follows: $$\frac{\left(g+\epsilon y,H(g+\epsilon y)\right)}{\left(g+\epsilon y,M(g+\epsilon y)\right)}>c$$ $$\left(g+\epsilon y,H(g+\epsilon y)\right)-с\left(g+\epsilon y,M(g+\epsilon y)\right)>0$$ Now we have to make sure we exhaust the entire $A$ in the following argument. A simple construction is suggested by the Gram-Smidt orthogonalization process, namely, let $$y=h-(h,Mf)f$$ with normalization condition $(f,Mf)=1$. Indeed $$(y,Mf)=(h,Mf)-(h,Mf)(f,Mf)=0$$ Expanding, for example the expression of the numerator, bearing in mind that $H$ is self-adjoint, we obtain: $$\left(g+\epsilon y,H(g+\epsilon y)\right)=(g,Hg)+2\epsilon (y,Hg)+\epsilon^2(h,h)=c(g,Mg)+2\epsilon(y,Hg)+\epsilon^2(y,Hy)$$ Now $$(y,Hg)=(h,Hg)-(f,Hg)(y,Mf)=(h,Hg)$$ Performing the same for the denominator and subtracting: $$\left(g+\epsilon y,H(g+\epsilon y)\right)-с\left(g+\epsilon y,M(g+\epsilon y)\right)=2\epsilon\left( (h,Hg-cMg)\right)+\epsilon^2\left((y,Hy)-c(y,My)\right)$$ Since $h$ is now ranges over $X$, the only way for the RHS to remain positive is for $$Hg=cMg$$ to hold.

Note: The above argument can be made slightly more concise if scalar product on $X$ is redefined in terms if $M$, that is $\left<x,y\right>=(x,My)$

-
why is that for any $y\in A$, $y$ can be written as $y=h-(h,f)f$, while your $h$ here in any vector in $X$? –  henryforever14 Aug 1 '12 at 8:43
Ok, I think I implicitly redefined the scalar product here. Will edit the answer to clarify –  Valentin Aug 1 '12 at 11:49
Thank you! I didn't check it in detail, but looks right to me. And about that change of scalar product, that's similar to the answer Will Jagy gave. but since you are the only one answering the problem in a direct way, i will pick your answer as the best one. Thank you! –  henryforever14 Aug 1 '12 at 15:16

Find a Cholesky type decomposition of $M$ as $$M = L L^T$$ For our purpose, it does not matter that $L$ is lower triangular, any such decomposition will do. Take $$E = \left( L^T \right)^{-1},$$ so that $$E^T M E = I,$$ and $$M = \left( E^T \right)^{-1} E^{-1}.$$ Write your Rayleigh quotient as $$\frac{x^T H x}{x^T M x}$$ for column vectors $x.$ Now, make the (invertible) substitution $$x = E z,$$ the quotient becomes $$\frac{z^T E^T H E z}{z^T E^T M E z} = \frac{z^T (E^T H E) z}{z^T z}$$ So, define $$G = E^T H E$$ and we have $$\frac{x^T H x}{x^T M x} = \frac{z^T G z}{z^T z}$$ so we are just asking about eigenvectors of $G,$ which are pairwise orthogonal, call them $z_j$. If we take the first optimum, call it $z_1,$ your $$f = E z_1.$$ You wanted to next have $$g^T M f = 0.$$ However, if we simply take $$g = E z_2,$$ then $$g^T M f = z_2^T E^T \left( E^T \right)^{-1} E^{-1} E z_1 = z_2^T z_1 = 0.$$

-
Yes, that's what I thought. Change the generalized Rayleigh Quotient back to the standard one. But it seems a bit tedious. Is there a way to show this directly? –  henryforever14 Mar 17 '12 at 22:16