Orthogonal Matrices?

Question

The columns of matrix $\matrix M$ are orthogonal to each other

What does this statement mean? Aren't columns parallel to each other? How can columns be orthogonal- aren't columns parallel to each other vertically by definition?

Answer 1

Two vectors $x$ and $y$ are said to be orthogonal if

$$x\cdot y = 0$$

where $\cdot$ is the standard dot product on vectors:

$$x\cdot y = x_1y_1 + \cdots + x_n y_n$$

When someone says that the columns of a matrix are orthogonal, they mean that if you consider each of the columns to be a vector, those vectors are all orthogonal to each other.

In a little more detail, an $n\times n$ matrix can be viewed as $n$ vectors stacked vertically next to each other. Call the vectors $v_1$, ..., $v_n$. Then if the columns are orthogonal, we have

$$v_i \cdot v_j = 0$$

for every $i$ and $j$.