19
$\begingroup$

I know that a unitary matrix is a matrix whose inverse equals its conjugate transpose (or that multiplying it by its conjugate transpose yields the identity), but I don't have a deep intuition about it (I just accept the definition). So for example, when I encounter the statement that the left and right singular decompositions $U$ and $V$ in the SVD are unitary, I don't get the significance. I would appreciate if somebody could enlighten me to connect the dots and how to feel when encountering unitary matrices. I have the feeling there is something unwritten that I'm missing.

$\endgroup$
3
  • 1
    $\begingroup$ I think what you are missing is $\langle U v, U w\rangle= \langle v, w\rangle$, with $\langle \cdot,\cdot\rangle $ an inner product. This fact is definitely not unwritten but rather the basis independent definition of a unitary linear transformation. $\endgroup$
    – Fabian
    Sep 28, 2012 at 22:43
  • $\begingroup$ I didnt know there was a wikipedia site dedicated to unitary matrices. these properties def. helped. thanks $\endgroup$ Sep 28, 2012 at 23:25
  • $\begingroup$ In quantum mechanics we consider unitary transformations because they leave the bracket of quantum states unaltered. The bracket represents probabilities of measurement so the idea is roughly that swapping your current set of states by a new set created by a unitary transformation just gives you a new picture of the same quantum system. The new states still produce all the same physics. $\endgroup$ Feb 6, 2014 at 22:12

3 Answers 3

18
$\begingroup$

Structurally, unitary matrices are rotations and reflections. Perhaps it's more clear to first picture unitary diagonalization before the singular value decomposition. Suppose we unitarily diagonalize $$A = UDU^{\dagger}$$ In unitary diagonalization, we first rotate (and possibly reflect) from our standard basis into our new orthonormal basis. This is the action of $U^{\dagger}$. Then we perform stretches by the magnitudes of the eigenvalues in the respective basis directions. This is the action of the diagonal matrix $D$. Finally we rotate back to our original basis, which is the action of $U$ which reverses $U^\dagger$.

The action of a singular value decomposition is virtually identical, except that the "diagonal" matrix $\Sigma$ does not necessarily map the same space to itself, so that the rotations happen in different vector spaces.

$\endgroup$
5
  • $\begingroup$ Very clear, thank you. Just one question. When you say "stretch by the magnitudes of the eigenvectors," don't you mean "stretch by the eigenvalues"? If I understand correctly, all vectors on the same direction of an eigenvector are also eigenvectors, so what we need is 1. a unitary matrix (which has unitary eigenvectors as orthonormal basis) and 2. a diagonal matrix of eigenvalues. Is this correct? $\endgroup$
    – Tobia
    Jan 24, 2016 at 17:08
  • 1
    $\begingroup$ @Tobia Yes, you are absolutely correct that we stretch by the magnitude of the eigenvalues, in the direction of the corresponding eigenvectors. As you correctly pointed out, all vectors in the same direction of an eigenvector (i.e. scalar multiples of an eigenvector) are also eigenvectors themselves. You are correct that we need a unitary $U$ and a diagonal matrix $D$ to characterize a matrix (more precisely, a normal matrix). Suppose we have a normal matrix $A$, so that $A$ is unitarily diagonalizable. Let $\mathbf{v}_j$ denote the $j$th eigenvector of $A$. $\endgroup$
    – EuYu
    Jan 25, 2016 at 2:36
  • 1
    $\begingroup$ ... Note that the action of $A$ on $\mathbf{v}_j$ is to just stretch it by the corresponding eigenvalue $\lambda_j$. Since $A$ is normal, we know that the eigenvectors can be chosen to be orthonormal. Now, as usual, we form the matrix $U$ where the $j$th column is the vector $\mathbf{v}_j$. The action of $U$ is to take the $j$th standard basis vector $\mathbf{e}_j$ to the eigenvector $\mathbf{v}_j$, i.e. it is an orthogonal transformation which rotates the standard basis to the eigenbasis. If we apply the inverse $U^\dagger$ first, we rotate the eigenbasis to the standard basis. $\endgroup$
    – EuYu
    Jan 25, 2016 at 2:39
  • 1
    $\begingroup$ ... Since $\mathbf{v}_j$ has been mapped to $\mathbf{e}_j$ by $U^\dagger$, we need to stretch $\mathbf{e}_j$ by $\lambda_j$ since that's the original action of $A$. This is implemented by the diagonal matrix $D$. Now we rotate back to get the original matrix $A$. This sequence of actions completely characterizes any diagonalizable matrix, with the only difference being that normal matrices have orthogonal eigenvectors while other diagonalizable matrices may not necessarily have orthogonal eigenvectors. $\endgroup$
    – EuYu
    Jan 25, 2016 at 2:42
  • $\begingroup$ Great. Thanks again. $\endgroup$
    – Tobia
    Jan 25, 2016 at 11:52
14
$\begingroup$

Unitary matrices are the complex analogues of orthogonal matrices, and both are very common in the theory of Lie groups and Lie algebras. Orthogonal matrices are the matrix representations of real linear maps that preserve distance. Unitary matrices are the complex versions, and they are the matrix representations of linear maps on complex vector spaces that preserve "complex distances".

If you have a complex vector space then instead of using the scaler product like you would in a real vector space, you use the Hermitian product. The Hermitian product of two complex vectors (thought of as $n$-by-$1$ matrices), say $v$ and $w$, is defined to be $\langle v,w\rangle = \overline{v}^{\top}\! w$.

Consider an $n$-by-$n$ matrix, say $M$, with complex entries acting on $\mathbb{C}^n$. The matrix preserves the Hermitian product if and only if $\langle Mv,Mw\rangle = \langle v,w\rangle$:

$$\langle Mv, Mw \rangle = \langle v,w\rangle \iff \overline{(Mv)}^{\top}(Mw) = \overline{v}^{\top}w \iff \overline{v}^{\top}(\overline{M}^{\top}\!\! M)w = \overline{v}^{\top}w \, .$$ Thus, $M$ preserves the Hermitian product if and only if $\overline{M}^{\top}\!\! M$ is the $n$-by-$n$ identity matrix, i.e. $M$ is a unitary matrix.

$\endgroup$
1
  • $\begingroup$ upvoted. i wasnt familiar with orthogonal matrices and the preservation of distances. thanks! $\endgroup$ Sep 28, 2012 at 23:27
3
$\begingroup$

The unitary matrices are precisely those matrices which preserve the Hermitian inner product.

$$\langle v,w \rangle = \langle Uv,Uw \rangle$$

Hence it preserves the distance or length of a vector in the Unitary space (the finite-dimensional vector space over the complex number field $\mathbb{C}$ with an inner product) under rotation or reflection.

$\endgroup$
2
  • $\begingroup$ @Fantini thanks! $\endgroup$
    – gundamlh
    Feb 6, 2014 at 17:24
  • $\begingroup$ Is there a simple proof that preserving the inner product preservers length of the vectors? Widening the angle between two vectors (imagining the case where the product is > 0) but compensating that by lenghtening one of the vectors, also preserves the inner product, so it would be reassuring to see an argument why preserving the lengths follows from preserving the inner product. $\endgroup$
    – GolDDranks
    Aug 19, 2020 at 1:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.