Let $X\in\mathbb{R}^{n\times p}$ denote a matrix with $p$ linearly-independent columns, and let $L\in\mathbb{R}^{n\times n}$ denote a symmetric matrix. A generalized eigenvector problem to $(A, B)$ is formulated as a solution $v$ to $$Av=\lambda\cdot Bv,$$ where $v$ denotes a generalized eigenvector, and $\lambda$ denotes a generalized eigenvalue.

Now, suppose a diagonal matrix $D\in\mathbb{R}^{n\times n}$ is given. My question is: could the solution (eigenvector $v$) to $(X^TLX, X^TDX)$, i.e., $$X^TLXv=\lambda\cdot X^TDXv,$$ be obtained by first making columns of $X$ $D-$orthogonal, ie, $(X')^TDX'=I$, and then solving the plain eigenvector problem $(X')^TLX'\cdot v=\lambda\cdot v$?


I am not entirely certain why you call this a short-cut solution except maybe for the possibility of reducing the dimension of the problem when singular matrices are involved, but I believe such a solution would go as follows.

If this:

$$X^TLXv=\lambda\cdot X^TDXv $$

Then we have:

$$ X^T(L-\lambda D)Xv=0$$

If the columns of $X$ can be made D-orthogonal such that for some $Y \in \mathbb{R}^{p \times p}$

$$Y^T X^T \cdot D \cdot X \cdot Y = I$$

Then the equation may be transformed with left multiply by $Y^T$:

$$ Y^T X^T(L-\lambda D)X \underbrace{ Y Y^{-1}}_Iv=Y^T \cdot 0 = 0$$ $$ \left[(Y^T X^T)L(XY)-\lambda I \right] Y^{-1}v=0 $$

Substituting $Z=XY$ and $u=Y^{-1}v$ it becomes the regular symmetric eigenvector equation: $$(Z^TLZ-\lambda I)u=\mathbf 0$$

It is then necessay to find Y that diagonalizes the given the $p \times p$ symmetric matrix $Q=X^TDX$ so that $$Y^TQY=I$$ This is possible if Q is not singular. Assuming for now that $Q$ is not singular, from the symmetry of Q (and I am also supposing that the entries are not complex), there exists a non-singular normal matrix $P$ such that $PP^T=I$ and $P^TQP=D_Q$ with $D_Q$ diagonal. Once such $P$ and $D_Q$ are found, to find $Y$ one need only scale for the diagonal elements of $D_Q$ symmetrically from left multiple and right multiple by a diagonal matrix. Symmetry is then preserved, but if complex values are required to scale the $D_Q$ in such a symmetric manner, the resulting eigensystem equation becomes complex, and loses the property of being Hermitian. Otherwise, the system remains real and symmetric.

As for the question of the rank of $Q$, I believe it is not singular so long as the columns of $X$ are independent and the original diagonal matrix $D$ has no zero entries. At this time I have no good argument for that statement and invite anyone to show why it is or why it is not true.

  • $\begingroup$ This is a bit unclear to me (though, I like the manipulation!). So, given that $X$ is transformed to $X'$, such that $(X')^TDX'=I$, a solution $v$ to $X^TLX\cdot v=\lambda\cdot X^TDX\cdot v$ is a solution $u$ to $(X')^TLX'\cdot u=\lambda\cdot u$? Or, is $u=Y^{-1}v$? If the latter is correct, please correct your concluding paragraph ("the answer is yes"). $\endgroup$
    – user506901
    Oct 3 '12 at 15:18
  • 1
    $\begingroup$ Yes the latter looks correct to me, but since $u$ is found first, to get $v$ it would be $v=Y u$ $\endgroup$
    – adam W
    Oct 3 '12 at 15:47
  • $\begingroup$ Thanks; so the "answer is yes" does not apply, but "yes with additional application of a matrix". One more thing: given independent columns of $X\in\mathbb{R}^{n\times p}$, how do you guarantee that there is $Y\in\mathbb{R}^{p\times p}$ that makes $(X')TDX'=I$ for $X'=XY$? Moreover, how to guarantee that such $Y$ is invertible? $\endgroup$
    – user506901
    Oct 3 '12 at 15:53
  • $\begingroup$ See my latest edit, hope that helps. $\endgroup$
    – adam W
    Oct 4 '12 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.