Why is the inverse of an orthogonal matrix equal to its transpose? I don't get why that's the case. Or is it a definition? The way the concept was presented to me was that an orthogonal matrix has orthonormal columns. And that's it. 
 A: Well, if $ Q $ is any $ n \times n $ matrix with real entries with column vectors $ v_1, v_2, \ldots, v_n $, we have
$$ (Q^T Q)_{ij} = \sum_{k=1}^n Q^T_{ik} Q_{kj} = \sum_{k=1}^n Q_{ki} Q_{kj} = \sum_{k=1}^n (v_i)_k (v_j)_k = \langle v_i, v_j \rangle $$
Now, assume that $ Q $ is an orthogonal matrix, i.e its column vectors $ v_i $ are orthonormal. What does this tell you about the entries of $ Q^T Q $?
A: Let $A$ be an $n\times n$ matrix with real entries. The matrix $A$ is orthogonal if the column and row vectors are orthonormal vectors. In other words, if $v_1,v_2,\cdots,v_n$ are column vectors of $A$, we have
$v_i^Tv_j=\begin{cases}1 \quad\text{if }i=j\\ 0\quad\text{if } i\neq j\end{cases}$
If $A$ is an orthogonal matrix, using the above information we can show that $A^TA=I$. Since the column vectors are orthonormal vectors, the column vectors are linearly independent and thus the matrix $A$ is invertible. Thus, $A^{-1}$ is well defined.
Since $A^TA=I$, we have $(A^TA)A^{-1}=IA^{-1}=A^{-1}$. Since matrix multiplication is associative, we have $(A^TA)A^{-1}=A^T(AA^{-1})$, which equals $A^T$. We therefore have $A^T=A^{-1}$.
