I'm just reading a book on econometrics and now I'm stuck with a problem:

There is a Theorem on "Orthogonal Partitioned Regression" which says:

"In the multiple linear least squares regression of $y$ on two sets of variables $X_1$ and $X_2$, if the two sets of variables are orthogonal, then the separate coefficient vectors can be obtained by separate regressions of $y$ on $X_1$ alone and $y$ on $X_2$ alone. ..."

Furthermore, the authors says that $X_1$ and $X_2$ are orthogonal if: $$ (X_1)' X_2=0 \, , $$ which confuses me because so far I always thought the condition for orthogonality between two matrices is that $(X_1)' X_2=I$.

The book is: W. Greene, Econometric Analysis, 7th ed. / Theorem 3.1 / p.33

What I found so far is:

Link 1 and Link 2 (search for: "imagine"), but neither really helps me with my problem.

BR Fabian


The definitions I see in your links are unusual to me, but I think you have already answered your own question.

In the sentence beginning with "Imagine" sentence in your Link 2, I think this is the definition of "orthogonal matrices" that you want: Two matrices $X_{1}, X_{2}$ are orthogonal provided that "the columns of $X_{1}$ are orthogonal to the columns of $X_{2}$ such that $X_{1}'X_{2} = 0$."

I have not heard of a notion of "orthogonal matrices" outside of this question. An orthogonal matrix is a matrix $X$ satisfying $X'X = I$.

  • $\begingroup$ Thank you for your answer. Indeed, the author later on specifies and says: "... if the columns of X are orthogonal ..." and therefore (X1)'(X2)=0 is fine. However, according to the definition of wikipedia the definition of orthogonality is (X1)'(X2)=I. So I wonder what is the difference between two matrices being orthogonal and the columns of two matrices being ortohogonal. $\endgroup$ – Fabian Aug 19 '14 at 17:46
  • 1
    $\begingroup$ @Fabian No, that's not quite what Wikipedia says. Note that there's only a single matrix in the defining equation $Q^T Q = I$ in the article you linked. In your definition, there are 2 different matrices. $\endgroup$ – André 3000 Aug 19 '14 at 18:48

Two vectors $v,w \in \mathbb{R}^n$ are orthogonal iff $v^t w = 0$ where $t$ indicates the transpose. Really, we're using the dot product given by $\langle v, w \rangle = v^t w$.

There is a different notion of orthogonality for matrices. Here's one definition of an orthogonal matrix: $O \in \rm{M}_n(\mathbb{R})$ is orthogonal if $O^t O = I$. Equivalently, this means that the columns of $O$ are orthonormal, i.e., that they are orthogonal and have length $1$.

However, note that only one matrix appears in this second definition. We just say that $A$ is orthogonal or not, not that $A$ is orthogonal to $B$. So this is not the definition used by the author of your book.

  • $\begingroup$ Thank you for your answer. But, given these definitions, shouldn't it be (X1)'(X2)=I in the original example? X1 and X2 are matrices, not vectors (If I got it right.). At least if it is really about orthogonality of the matrices. Could it be that the differnce between (X1)'(X2)=0 and (X1)'(X2)=I that if "...=I" then X1 and X2 are the equal matrices while "...=0" can only be if they are not equal? $\endgroup$ – Fabian Aug 19 '14 at 18:30
  • 1
    $\begingroup$ No, I think the author just wants to say that the columns of $X_1$ and the columns of $X_2$ are orthogonal as vectors in $\mathbb{R}^n$, as in the first part of my answer. $\endgroup$ – André 3000 Aug 19 '14 at 18:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.