Let's assume we have three collinear variables/factors, $X_1$, $X_2$, $X_3$. Would there be a method to orthogonalize these variables in a simultaneous way: in other word, orthogonalizing them in such a way that the order of the variables doesn't matter?

To clarifi this, let's consider the following orthogonalization method where the order of the orthogonalization process actually matters:

Let's choose factor $X_1$. Now, we will regress $X_2$ with respect to $X_1$:

$X_2 = \alpha_3 + \beta^{(2)} * X_1 + \epsilon_2$

In this case, $\epsilon_2$ is the "pure" $X_2$ value: we have striped out any $X_1$ effect from $X_2$.

Now, we will orthogonalize $X_3$ with respect to the first two factors, $X_1$ and the pure factor $\epsilon_2$:

$X_3 = \alpha_2 + \beta^{(3,1)} * X_1 + \beta^{(3,2)} * \epsilon_2 + \epsilon_3$

Now, we obtain three orthogonal factors: $X_1$, $\epsilon_2$, and $\epsilon_3$.

However, the problem with this hierarchical methodology is that the order in which your orthogonalize (in this case $X_1$ first, then $X_2$, etc.) matters. In this case, $X_1$ has the most effect, following by $\epsilon_2$, then finally $\epsilon_3$.

Is there any methodology that would allow all variables to be orthogonalized with respect to each other, but in a simultaneous way, rather than hierarchically?

Thanks in advance!



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.