Eigenvalue decomposition of block covariance matrix for Canonical Correlation Analysis (CCA) Edited:
My question is related to a tutorial I was reading.
The covariance matrix is a block matrix where $C_{xx}$ and $C_{yy}$ are within-set covariance matrices and $C_{xy} = C_{yx}^T$ are between-sets covariance matrices.
$$
  \left[\begin{array}{r r}
    C_{xx} & C_{xy}\\
    C_{yx} & C_{yy}
  \end{array}\right]
$$
The tutorial says that the canonical correlations between $x$ and $y$ can be found by solving the eigenvalue equations
$$
  C_{xx}^{-1}C_{xy}C_{yy}^{-1}C_{yx} \hat w_x = \rho^2 \hat w_x \\
  C_{yy}^{-1}C_{yx}C_{xx}^{-1}C_{xy} \hat w_y = \rho^2 \hat w_y 
$$
where the eigenvalues  are the squared canonical correlations and the eigenvectors  and  are the normalized canonical correlation basis vectors.
What I do not understand is how the eigenvalue equations are found by using the covariance matrix? Can someone please explain how we get those sets of equations?
Thanks.
 A: Canonical correlation between two random vectors $X$ and $Y$ is obtained as the maximal correlation between $a^TX$ and $b^TY$, where the maximum is taken over vectors $a$ and $b$. We can assume without loss of generality that $a^T \Sigma_x a = b^T \Sigma_y b = 1$. Assume for simplicity also that $E(X) = 0$ and $E(y) = 0$. The the correlation between $a^TX$ and $b^TY$ is just 
$$E(a^T X)(b^TY) = E(a^T X)(Y^Tb) = a^T E(XY^T) b = a^T \Sigma_{xy}b.$$
You can now use either Lagrange duality or Cauchy-Schwarz.
Say we use Lagrange duality. The optimal should maximize
$$a^T \Sigma_{xy}b -\frac12\mu (a^T \Sigma_x a) - \frac12\lambda (b^T \Sigma_y b)$$
over $a$ and $b$. ($\frac12$s in the above are for convenience.) Differentiating with respect to $a$ and $b$ gives
$$
\begin{align*}
\Sigma_{xy} b - \mu \Sigma_x a &= 0 \\
\Sigma_{yx} a-  \lambda \Sigma_y b &= 0,
\end{align*}
$$
Multiplying the first by $a^T$ and the second by $b^T$ and enforcing the constraints shows that $\mu = \lambda$. Then, if $\Sigma_x$ and $\Sigma_y$ are invertible you can solve the equations for what you have. That is,
$$
\begin{align*}
\Sigma_x^{-1} \Sigma_{xy} b - \mu  a &= 0 \\
\Sigma_y^{-1} \Sigma_{yx} a-  \mu b &= 0,
\end{align*}
$$
implying
$$
\begin{align*}
\frac{1}{\mu} \Sigma_x^{-1} \Sigma_{xy}  \Sigma_y^{-1} \Sigma_{yx}  a - \mu  a &= 0
\end{align*}
$$
A: You can find the eigenvalue equation from the CCA's Lagrangian form. Hope it help.
PS: I don't know how to write latex in reply:( 
