# Geometric Interpretation of Eigendecomposition

If we have an orthogonal matrix $$U$$, then $$U^Tx$$ is essentially a rotation of the vector $$x$$. If we have a diagonal matrix $$\Lambda$$, then $$\Lambda x$$ is scaling the vector $$x$$ in each direction by the corresponding diagonal value.

Since any symmetric matrix $$S=U\Lambda U^T$$, $$Sx=U\Lambda U^Tx$$ which is rotation of $$x$$ by some angle $$\vartheta$$, scaling it by $$\Lambda$$ and then re-rotating by angle $$-\vartheta$$. Does this imply that multiplying by any symmetric matrix $$S$$ is just a scaling by its eigenvalues since the net rotation is $$0$$?

• Do you possibly mean orthogonal/unitary instead of idempotent? Jun 11, 2018 at 21:24
• A visualization I found helpful: A unit circle will be transformed into an ellipse with the length of the major axis equal to the larger eigenvalue and the orientation with the corresponding eigenvector, and the minor axis similarly associated with the smaller eigenvalue and and eigenvector. Jun 11, 2018 at 22:24

Yes, but the orthogonal matrix $U$ determines in which directions this scaling happens. Only applying the diagonal matrix

$$\Lambda=\begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$$

scales by two on the $x$-axis and by three on the $y$-axis.

But you can scale along other (orthogonal) axes if you want to by adding a rotation matrix.

You can imagine this by first transforming the coordinate system by $U^T$, so that the desired stretching axes line up with the coordinate axes (red and blue lines in the image), then you scale along the usual coordinate axes by $\Lambda$, and then you rotate back by $U$.

• Does this mean that the non-uniform scaling along the different directions can be interpreted as another rotation? Since the $\Lambda$ mentioned in your example would scale the vector $(x_1,y_1)$ to $(2x_1,3y_1)$ and the latter has a different angle wrt $x$ axis. Jun 11, 2018 at 22:38
• @AmritPrasad I am not sure if I understood your question correctly. A rotation is necessary to non-uniformly stretch along non-coordinate axes, but I would not say that such a transformation can be seen as a rotation (as you cancel it out by applying both, $U$ and $U^T$). Jun 12, 2018 at 8:10
• If we are measuring the angle of a vector $(x_1,y_1)$, then it's going to be $tan^{-1}(\frac{y_1}{x_1})$ wrt to the x-axis. Upon scaling each of these coordinates non-uniformly, the angle is going to become $tan^{-1}(\frac{3y_1}{2x_1})$. This implies a rotation in the original coordinate axes. The only way this wouldn't happen is if $\Lambda=kI$ where $I$ is the identity matrix and $k$ is a constant. My point is that just a multiplication by $\Lambda$ can be constituted as a rotation in the sense that the angle defined as $tan^{-1}(y/x)$ is going to change. Jun 12, 2018 at 19:24
• @AmritPrasad I see. Yes, any specific vector will "rotate" (and scale) when we apply any linear transformation. But every vector might rotate differently. If the rotation of all vectors is the same and no scaling happens, then the linear transformation is a rotation (matrix). Otherwise it is not. In the case of $\Lambda$ above, it is no (global) rotation because the vector $(1,0)$ will not change direction, while $(1,1)$ will do so. So the change of angle is not the same for all vectors. Jun 12, 2018 at 19:55
• Yeah. Thanks for your help! Really cleared up a lot of things. Jun 13, 2018 at 1:54

General case for a diagonizable matrix

$$S = U.\Lambda.U^{-1} = \begin{pmatrix} 3 & 1 \\ 0 & 2 \end{pmatrix} =\begin{pmatrix} 1 & -1 / \sqrt 2 \\ 0 & 1 / \sqrt 2 \end{pmatrix} \begin{pmatrix} 3 & 0 \\ 0 & 2 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ 0 & \sqrt 2 \end{pmatrix}$$

Multiplying by $$S$$ is equivalent to multiplying successively by $$U^{-1}$$, $$\Lambda$$ and $$U$$.

• Matrix $$U$$ is made of columns which are the eigenvectors of $$S$$. Multiplying by $$U^{-1}$$ aligns the eigenvectors with the basis vectors.

• Scaling by $$\Lambda$$ preserves the orientation of the eigenvectors, as they are aligned with the basis.

• The scaled, but with unchanged orientation, eigenvectors are returned to their initial orientation by multiplying by $$U$$.

The eigenvectors: $$v1=\begin{pmatrix} 1 \\ 0 \end{pmatrix} \text { and } v2=\begin{pmatrix} -1 / \sqrt 2 \\ 1 / \sqrt 2 \end{pmatrix}$$ are shown in color.

Specific case of a symmetric matrix

In the case of a real symmetric matrix $$S$$, $$U$$ is orthogonal. $$U^{-1}=U^T$$ and the relationship can be written:

$$S = U.\Lambda.U^T$$

$$U$$ is a unitary matrix, in the general case, a rotation matrix.

$$S = \begin{pmatrix} 3 & \sqrt 2 \\ \sqrt 2 & 2 \end{pmatrix} =\begin{pmatrix} \sqrt {2/3} & -1 / \sqrt 3 \\ 1 / \sqrt 3 & \sqrt {2/3} \end{pmatrix} \begin{pmatrix} 4 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} \sqrt {2/3} & 1 / \sqrt 3 \\ -1 / \sqrt 3 & \sqrt {2/3} \end{pmatrix}$$

Does this imply that multiplying by any symmetric matrix $$S$$ is just a scaling by its eigenvalues.

Yes, but as shown, the scaling must be applied in the basis defined by the eigenvectors, or equivalently the basis can be changed prior to apply it. In the case of a real symmetric matrix, the eigenvectors are orthogonal.