0
$\begingroup$

Let $A$ and $B$ be $n \times n$ symmetric matrices with real entries and let $k \geq 2$ be an integer.

I want to find the maximum of $$ \frac{\sum_{i=1}^k X_i^{\mathrm T}\,A\,X_i}{\sum_{i=1}^k X_i^{\mathrm T}\,B\,X_i} $$ where the unknowns $X_i$ are vectors, under the constraints that the $X_i$ are normalized ($X_i^{\mathrm T}\,X_i = 1$ for all $i$) and orthogonal ($X^{\mathrm T}_i\,X_j = 0$ if $i \neq j$).

The problem I am facing (and that is why I add the orthogonal constraints) is that, classically, one will get the eigenvectors of $B^{-1}\,A$ (see here and here) but there is no reason that these eigenvectors will be orthogonal (since $B^{-1}\,A$ has no reason to be symmetric). Maybe I am missing something...

$\endgroup$

1 Answer 1

2
$\begingroup$

Assuming that $B$ p.s.d., note that $B^{-\frac 1 2} A B^{-\frac 1 2}$ is symmetric, so it has $n$ eigenvectors $y_1,\dots,y_n$ that are orthogonal. Then if you consider $x_i=B^{-\frac 1 2}y_i$, then $x_1,\dots x_n$ are $n$ eigenvectors of $B^{-1}A$: $$B^{-1}A x_i = B^{-\frac 1 2} \left ( B^{-\frac 1 2} A B^{-\frac 1 2}\right)B^{\frac 1 2}x_i = \lambda_i B^{-\frac 1 2}y_i=\lambda_i x_i$$ Those eigenvectors are not orthogonal w.r.t. the regular inner product. That's because: $$x_i^Tx_j = y_i^TB^{-\frac 1 2}B^{-\frac 1 2}y_j=y_i^TB^{-1}y_j$$ But they are orthogonal under the inner product defined by $B$: $$x_i^TBx_j = x_i^TB^{\frac 1 2}B^{\frac 1 2}x_j=y_i^Ty_j$$

$\endgroup$
2
  • $\begingroup$ Thanks! So, if I understand correctly, in PCA, we get eigenvectors that are orthogonal not with the regular inner product but with the inner product defined by $B$? $\endgroup$
    – Héhéhé
    May 14 at 5:03
  • $\begingroup$ For the regular PCA, matrix $A$ is the covariance matrix of your (mean-centered) data set, and matrix $B$ is the identity. So your eigenvectors are orthogonal with the regular inner product. $\endgroup$ May 14 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.