Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top


My question is related to a tutorial I was reading.

The covariance matrix is a block matrix where $C_{xx}$ and $C_{yy}$ are within-set covariance matrices and $C_{xy} = C_{yx}^T$ are between-sets covariance matrices.

$$ \left[\begin{array}{r r} C_{xx} & C_{xy}\\ C_{yx} & C_{yy} \end{array}\right] $$

The tutorial says that the cannonical correlations between $x$ and $y$ can be found by solving the eigenvalue equations

$$ C_{xx}^{-1}C_{xy}C_{yy}^{-1}C_{yx} \hat w_x = \rho^2 \hat w_x \\ C_{yy}^{-1}C_{yx}C_{xx}^{-1}C_{xy} \hat w_y = \rho^2 \hat w_y $$

where the eigenvalues are the squared canonical correlations and the eigenvectors and are the normalized canonical correlation basis vectors.

What I do not understand is how the eigenvalue equations are found by using the covariance matrix? Can someone please explain how we get those sets of equations?


share|cite|improve this question
up vote 1 down vote accepted

Canonical correlation between two random vectors $X$ and $Y$ is obtained as the maximal correlation between $a^TX$ and $b^TY$, where the maximum is taken over vectors $a$ and $b$. We can assume without loss of generality that $a^T \Sigma_x a = b^T \Sigma_y b = 1$. Assume for simplicity also that $E(X) = 0$ and $E(y) = 0$. The the correlation between $a^TX$ and $b^TY$ is just $$E(a^T X)(b^TY) = E(a^T X)(Y^Tb) = a^T E(XY^T) b = a^T \Sigma_{xy}b.$$

You can now use either Lagrange duality or Cauchy-Schwarz. Say we use Lagrange duality. The optimal should maximize $$a^T \Sigma_{xy}b -\frac12\mu (a^T \Sigma_x a) - \frac12\lambda (b^T \Sigma_y b)$$ over $a$ and $b$. ($\frac12$s in the above are for convenience.) Differentiating with respect to $a$ and $b$ gives $$ \begin{align*} \Sigma_{xy} b - \mu \Sigma_x a &= 0 \\ \Sigma_{yx} a- \lambda \Sigma_y b &= 0, \end{align*} $$ Multiplying the first by $a^T$ and the second by $b^T$ and enforcing the constraints shows that $\mu = \lambda$. Then, if $\Sigma_x$ and $\Sigma_y$ are invertible you can solve the equations for what you have. That is, $$ \begin{align*} \Sigma_x^{-1} \Sigma_{xy} b - \mu a &= 0 \\ \Sigma_y^{-1} \Sigma_{yx} a- \mu b &= 0, \end{align*} $$ implying $$ \begin{align*} \frac{1}{\mu} \Sigma_x^{-1} \Sigma_{xy} \Sigma_y^{-1} \Sigma_{yx} a - \mu a &= 0 \end{align*} $$

share|cite|improve this answer

You can find the eigenvalue equation from the CCA's Lagrangian form. Hope it help.

PS: I don't know how to write latex in reply:(

share|cite|improve this answer
Hi Dylan, you seem to be new here. For some basic information about writing math at this site see e.g. here, here, here and here. It's not always needed if answers are in the form of hints like this one is, but you may find that using LaTeX increases the readability of your answers and makes them easier to understand! – Tom Oldfield Dec 20 '12 at 4:24

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.