0
$\begingroup$

I have the following g[X] = (X.(A.X))/(X.X), where A is a symmetric matrix and the function g[X] takes in a vector X as it's argument.

When i used Mathematica to plot the Densityplot of g[X]Density Plot of g[x]

In the above Density plot, lighter the square, the higher g[x] is.

I then extracted the eigenvectors from A and plotted them as follows: Density plot with eigenvectors

What i noticed interesting was that the eigenvectors, apart from being orthogonal since they are from the symmetric Matrix A, also point towards what appears to be the maxima and minima of the density plot. Is there a reason why the eigenvectors point in the direction of maxima and minima of the density plot?

$\endgroup$

1 Answer 1

1
$\begingroup$

Is $g(x) = \frac{x^\top A x}{x^\top x}$? And I am assuming $x$ is a $2\times 1$ vector. Since $A$ is symmetric, we can write its decomposition as $A = Q\Lambda Q^\top$, where $Q$ is an orthonormal eigenvector matrix and $\Lambda$ is the diagonal matrix of eigenvalues. $$g(x) = \frac{x^\top (Q\Lambda Q^\top)x}{x^\top x} \\ = \frac{(Q^\top x)^\top \Lambda (Q^\top x)}{x^\top x} \\ = [x^\top q_1 \ x^\top q_2] \begin{bmatrix}\lambda_1 & 0 \\ 0 & \lambda_2 \end{bmatrix} \begin{bmatrix} q_1^\top x \\ q_2^\top x \end{bmatrix} \Big{/} x^\top x \\ = \frac{\lambda_1 x^\top q_1 q_1^\top x + \lambda_2 x^\top q_2 q_2^\top x}{x^\top x}$$

Now, assuming $\lambda_1 > \lambda_2$, $\max_x g(x)$ will occur when $x = q_1$, $$\max_x g(x) = \frac{\lambda_1 (q_1^\top q_1) (q_1^\top q_1) + \lambda_2 (q_1^\top q_2) (q_2^\top q_1)}{q_1^\top q_1}$$ Using the fact that $Q^\top Q = I\ \implies q_1^\top q_1 = 1, q_1^\top q_2 = 0$, we get $$\max_x g(x) = \lambda_1 \\ \arg\max_x g(x) = q_1$$ Similarly, $\min_x g(x)$ will occur when $x = q_2$. $$\min_x g(x) = \lambda_2 \\ \arg\min_x g(x) = q_2$$

This is why the eigenvectors point in the direction of the maxima and minima of the plot. This is also the theory behind Principal Components Analysis (PCA), which is used for dimensionality reduction by projecting onto the eigenvector associated with the maximum eigenvalue, in this case, $q_1$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .