Find the value of unit vector that maximizes the following cost function Let us consider the following cost function:
$$J(\mathbf{A},\mathbf{B}) = \frac{\mathbf{u}^T\mathbf{Au}}{\mathbf{u}^T\mathbf{Bu}}$$
where, $\mathbf{A}$ and $\mathbf{B}$ are given symmetric positive definite matrices and variable $\mathbf{u}$ is a unit vector. I want to know the steps for finding the value of $\mathbf{u}$ that maximizes $J$. Additionally, I have the following queries:


*

*Is maximizing $J$ equivalent to maximizing the numerator?

*Is maximizing $J$ equivalent to minimizing denominator?

*Is it possible to get two different answers for points 1 and 2?

 A: Define a few scalar variables and their differentials
$$\eqalign{
\alpha &= A:uu^T &\implies d\alpha = 2Au:du \cr
\beta  &= B:uu^T &\implies d\beta  = 2Bu:du \cr
}$$ where a colon denotes the trace/Frobenius product, i.e. 
$\,\,\,A\!:\!B={\rm tr\,}(A^TB)$
Write the cost function in terms of these, then find its differential and gradient
$$\eqalign{
 J &= \frac{\alpha}{\beta} \cr
dJ &= \beta^{-2}({\beta\,d\alpha-\alpha\,d\beta}) \cr
 &= 2\beta^{-2}({\beta Au-\alpha Bu}): du \cr
 &= 2\beta^{-1}(Au-JBu) : du \cr
\frac{\partial J}{\partial u} &= 2\beta^{-1}(Au-JBu) \cr
}$$
Set the gradient to zero and solve
$$\eqalign{
 Au &= JBu \cr
 (B^{-1}A)u &= Ju \cr
}$$
This is just an eigenvalue equation. So $J$ is the largest eigenvalue of $\,(B^{-1}A)$ and $u$ is the associated eigenvector.
A: No, it is not as simple as maximizing the numerator or minimizing the denominator separately. These will give different answers in general. $u^\top Au$ is maximized when $u$ is any eigenvector with maximal eigenvalue, while $u^\top Bu$ is minimized when $u$ is a minimal eigenvector. There is no reason these should be related, because $A$ and $B$ are unrelated.
Your program is
$$
\begin{aligned}
&\text{maximize}_u & &\frac{u^\top Au}{u^\top Bu}\\
&\text{subject to} & &u^Tu=1
\end{aligned}
$$
The constraint $u^Tu=1$ is unnecessary, since scaling $u$ does not affect $J(u)$. So we may drop this constraint, and instead impose the constraint $u^\top Bu=1$. This simplifies the problem since the denominator is now $1$, and we have the equivalent program
$$
\begin{aligned}
&\text{maximize}_u & &{u^\top Au}\\
&\text{subject to} & &{u^\top Bu}=1
\end{aligned}
$$
This can be solved via Lagrange multipliers, for example.
A: Since $\rm B$ is positive definite, it has an invertible square root. Let $\mathrm v := \mathrm B^{\frac 12} \mathrm u$. Hence,
$$\frac{\mathrm u^\top \mathrm A \,\mathrm u}{\mathrm u^\top \mathrm B \,\mathrm u} = \frac{\mathrm v^\top \mathrm B^{-\frac 12} \mathrm A \mathrm B^{-\frac 12} \mathrm v}{\mathrm v^\top \mathrm v} \leq \lambda_{\max} \left( \mathrm B^{-\frac 12} \mathrm A \mathrm B^{-\frac 12}\right) = \lambda_{\max} \left( \,\, \mathrm A \mathrm B^{-1} \right)$$
