7
$\begingroup$

I find it non-intuitive if I impose that all of a square matrix's columns are normalized and mutually orthogonal, then all its rows are also normalized and mutually orthogonal. Any intuitive explanation for this? Also if I relax the conditions to be only mutually orthogonal without being normalized, is this still true? And why so?

$\endgroup$
5
  • 1
    $\begingroup$ What? I think there is the word "rows" missing a few times. $\endgroup$
    – Jan
    May 26, 2020 at 10:52
  • $\begingroup$ Sorry I have corrected the typo. $\endgroup$
    – Sam
    May 29, 2020 at 3:54
  • $\begingroup$ @ArturoMagidin I already corrected the title so perhaps u can uncomment already? $\endgroup$
    – Sam
    Jun 30, 2020 at 7:49
  • 5
    $\begingroup$ It appears unintuitive because it is unintuitive. The said property is a consequence of the fact that $AB=I$ iff $BA=I$ for square matrices $A$ and $B$. It relies on the finiteness of the dimension of the vector space and has little to do with inner products or geometry. In an infinite-dimensional inner product space, it can happen that $A^\ast A=I$ but $AA^\ast\ne I$ (e.g. when $A$ is the right-shift operator in $\ell^2$ and $A^\ast$ is the left shift). $\endgroup$
    – user1551
    Nov 17, 2021 at 11:26
  • 2
    $\begingroup$ I'm doubling down on user1551's comment: It is critical that A have a finite size, and that A be square for the result to hold. $\endgroup$ Nov 18, 2021 at 11:27

3 Answers 3

2
+50
$\begingroup$

For a matrix $Q$ with normalized orthogonal columns, $Q^TQ=I,$ then $Q^T=Q^{-1}$ and $QQ^T=Q Q^{-1}=I.$ $Q$ also has normalized orthogonal rows.

$A=\begin{bmatrix}1&0&2\\ 1&-1&-1\\ 1&1&-1\end{bmatrix}$ clearly is a matrix with orthogonal columns, but without orthogonal rows.

Assume $A^TA=D,$ $D$ is an invertible diagonal matrix. Let $AP=B,$ $P$ is a diagonal matrix which normalized columns. So $B$ is an orthogonal matrix. Let's see how $AP$'s rows are also normalized, i.e. $AP=B=QA,$ how $Q$ looks like?

$$PDP=P^TA^TAP=B^TB=I\Rightarrow P^2=D^{-1}\quad (\text{normalization, of course})$$ $$QAA^TQ^T=BB^T=I \Rightarrow Q^TQ=(AA^T)^{-1}$$

So $Q$ may not be a diagonal matrix. Otherwise $AA^T$ is also diagonal for sure, which contradicts our counter-example. This means when we normalize our column spaces, row basis may change during the operation.

I don't know what we can get further. I remember that when I study linear algebra, I don't find strong connection between row spaces and column spaces, especially geometrical meaning. I hope someone has learned matrix theory can answer this question deeply.

$\endgroup$
2
$\begingroup$

I assume you mean orthonormal coulmn implies orthonormal rows. If so, than:

Let $Q$ be a square matrix with orthonormal columns. Therefore we know:

$$Q'Q=I \implies Q'=q^{-1}$$

$$(Q')'Q'=QQ'=I$$

Therefore, we have that the rows are also orthonormal.

$\endgroup$
3
  • $\begingroup$ Yes, when I write $Q'$ I mean transpose of matrix. $\endgroup$
    – Eminem
    May 26, 2020 at 11:12
  • $\begingroup$ Yea the math looks neat but is there some intuition? Also if the columns are not normalized is it still true? $\endgroup$
    – Sam
    May 29, 2020 at 3:52
  • $\begingroup$ The derivation above is incorrect. $Q'Q=I$ which follows $(Q'Q)'=I'$ $=>Q'Q=I$ which does not prove the point raised in question. To say that $Q'Q=I$ implies $(Q')'Q'=I$ is strictly incorrect. $\endgroup$ Apr 6, 2021 at 5:33
2
$\begingroup$

The derivation done by @eminem is mathematically incorrect. 𝑄′𝑄=𝐼 does not imply (𝑄′)′𝑄′=𝐼. Below is a solution that could answer the question

$Q^\top Q=I => (Q^\top Q)^{-1}=I^{-1} => Q^{-1}Q^{\top -1}=I => QQ^{-1}Q^{\top -1}=Q => QQ^{-1}Q^{\top -1}Q^\top =QQ^\top => I=QQ^\top$

This means that orthonormal rows/columns imply orthonormal columns/rows. Of course, here I assume that $Q$ is a full rank square matrix. This conclusion is not always true for rectangle matrices.

$\endgroup$
3
  • 1
    $\begingroup$ "mathematically incorrect" - I disagree. The logic in @eminem's proof is fine although it could have used a few words of explanation. $Q'Q = I$, therefore $Q'$ is a left inverse of $Q$. Since $Q$ is square, this implies that $Q'$ is the two-sided inverse of $Q$, and therefore $QQ' = I$. $\endgroup$
    – user169852
    Apr 6, 2021 at 5:55
  • $\begingroup$ @Bungo What you have written is absolutely correct. But (𝑄′)′𝑄′ written by eminem is misleading. $\endgroup$ Apr 6, 2021 at 6:00
  • $\begingroup$ I agree that it could have been stated more clearly. $\endgroup$
    – user169852
    Apr 6, 2021 at 6:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .