# Orthonormal columns implies orthonormal rows

I find it non-intuitive if I impose that all of a square matrix's columns are normalized and mutually orthogonal, then all its rows are also normalized and mutually orthogonal. Any intuitive explanation for this? Also if I relax the conditions to be only mutually orthogonal without being normalized, is this still true? And why so?

• What? I think there is the word "rows" missing a few times.
– Jan
May 26, 2020 at 10:52
• Sorry I have corrected the typo.
– Sam
May 29, 2020 at 3:54
• @ArturoMagidin I already corrected the title so perhaps u can uncomment already?
– Sam
Jun 30, 2020 at 7:49
• It appears unintuitive because it is unintuitive. The said property is a consequence of the fact that $AB=I$ iff $BA=I$ for square matrices $A$ and $B$. It relies on the finiteness of the dimension of the vector space and has little to do with inner products or geometry. In an infinite-dimensional inner product space, it can happen that $A^\ast A=I$ but $AA^\ast\ne I$ (e.g. when $A$ is the right-shift operator in $\ell^2$ and $A^\ast$ is the left shift). Nov 17, 2021 at 11:26
• I'm doubling down on user1551's comment: It is critical that A have a finite size, and that A be square for the result to hold. Nov 18, 2021 at 11:27

For a matrix $$Q$$ with normalized orthogonal columns, $$Q^TQ=I,$$ then $$Q^T=Q^{-1}$$ and $$QQ^T=Q Q^{-1}=I.$$ $$Q$$ also has normalized orthogonal rows.

$$A=\begin{bmatrix}1&0&2\\ 1&-1&-1\\ 1&1&-1\end{bmatrix}$$ clearly is a matrix with orthogonal columns, but without orthogonal rows.

Assume $$A^TA=D,$$ $$D$$ is an invertible diagonal matrix. Let $$AP=B,$$ $$P$$ is a diagonal matrix which normalized columns. So $$B$$ is an orthogonal matrix. Let's see how $$AP$$'s rows are also normalized, i.e. $$AP=B=QA,$$ how $$Q$$ looks like?

$$PDP=P^TA^TAP=B^TB=I\Rightarrow P^2=D^{-1}\quad (\text{normalization, of course})$$ $$QAA^TQ^T=BB^T=I \Rightarrow Q^TQ=(AA^T)^{-1}$$

So $$Q$$ may not be a diagonal matrix. Otherwise $$AA^T$$ is also diagonal for sure, which contradicts our counter-example. This means when we normalize our column spaces, row basis may change during the operation.

I don't know what we can get further. I remember that when I study linear algebra, I don't find strong connection between row spaces and column spaces, especially geometrical meaning. I hope someone has learned matrix theory can answer this question deeply.

I assume you mean orthonormal coulmn implies orthonormal rows. If so, than:

Let $$Q$$ be a square matrix with orthonormal columns. Therefore we know:

$$Q'Q=I \implies Q'=q^{-1}$$

$$(Q')'Q'=QQ'=I$$

Therefore, we have that the rows are also orthonormal.

• Yes, when I write $Q'$ I mean transpose of matrix. May 26, 2020 at 11:12
• Yea the math looks neat but is there some intuition? Also if the columns are not normalized is it still true?
– Sam
May 29, 2020 at 3:52
• The derivation above is incorrect. $Q'Q=I$ which follows $(Q'Q)'=I'$ $=>Q'Q=I$ which does not prove the point raised in question. To say that $Q'Q=I$ implies $(Q')'Q'=I$ is strictly incorrect. Apr 6, 2021 at 5:33

The derivation done by @eminem is mathematically incorrect. 𝑄′𝑄=𝐼 does not imply (𝑄′)′𝑄′=𝐼. Below is a solution that could answer the question

$$Q^\top Q=I => (Q^\top Q)^{-1}=I^{-1} => Q^{-1}Q^{\top -1}=I => QQ^{-1}Q^{\top -1}=Q => QQ^{-1}Q^{\top -1}Q^\top =QQ^\top => I=QQ^\top$$

This means that orthonormal rows/columns imply orthonormal columns/rows. Of course, here I assume that $$Q$$ is a full rank square matrix. This conclusion is not always true for rectangle matrices.

• "mathematically incorrect" - I disagree. The logic in @eminem's proof is fine although it could have used a few words of explanation. $Q'Q = I$, therefore $Q'$ is a left inverse of $Q$. Since $Q$ is square, this implies that $Q'$ is the two-sided inverse of $Q$, and therefore $QQ' = I$.
– user169852
Apr 6, 2021 at 5:55
• @Bungo What you have written is absolutely correct. But (𝑄′)′𝑄′ written by eminem is misleading. Apr 6, 2021 at 6:00
• I agree that it could have been stated more clearly.
– user169852
Apr 6, 2021 at 6:01