Can we define component of a matrix which is orthogonal to another matrix? Given two vectors $A$ and $B$ one can easily find component of $A$ along $B$ and component of $A$ perpendicular/orthogonal to $B$ and vice versa. This is possible as we can define dot product of two vectors. As I understand if $A$ and $B$ are matrix then our dot product does not make sense, does it? If not is there a way to define component and orthogonal component of a matrix with respect to the other? Or there is no such concepts for matrices?
 A: The Frobenius inner product of matrices $A, B \in \mathbb R^{m \times n}$ is defined by
$$\langle A, B\rangle := \mbox{tr} (A^T B) = \mbox{tr} (B A^T) = \mbox{tr} (A B^T) = \mbox{tr} (B^T A)$$
For example, the standard basis "vectors" for $\mathbb R^{2 \times 2}$ are
$$M_{11} := \begin{bmatrix} 1 & 0\\ 0 & 0\end{bmatrix} \qquad \quad M_{12} := \begin{bmatrix} 0 & 1\\ 0 & 0\end{bmatrix} \qquad \quad M_{21} := \begin{bmatrix} 0 & 0\\ 1 & 0\end{bmatrix} \qquad \quad M_{22} := \begin{bmatrix} 0 & 0\\ 0 & 1\end{bmatrix}$$
Note that
$$\langle M_{11}, M_{11}\rangle = 1 \qquad \qquad \langle M_{11}, M_{12}\rangle = 0 \qquad \qquad \langle M_{11}, M_{21}\rangle = 0 \qquad \qquad \langle M_{11}, M_{22}\rangle = 0$$
$$\langle M_{12}, M_{11}\rangle = 0 \qquad \qquad \langle M_{12}, M_{12}\rangle = 1 \qquad \qquad \langle M_{12}, M_{21}\rangle = 0 \qquad \qquad \langle M_{12}, M_{22}\rangle = 0$$
$$\langle M_{21}, M_{11}\rangle = 0 \qquad \qquad \langle M_{21}, M_{12}\rangle = 0 \qquad \qquad \langle M_{21}, M_{21}\rangle = 1 \qquad \qquad \langle M_{21}, M_{22}\rangle = 0$$
$$\langle M_{22}, M_{11}\rangle = 0 \qquad \qquad \langle M_{22}, M_{12}\rangle = 0 \qquad \qquad \langle M_{22}, M_{21}\rangle = 0 \qquad \qquad \langle M_{22}, M_{22}\rangle = 1$$
A: There is such a thing called orthogonal matrix in linear algebra. An orthogonal matrix is a square matrix with real entries whose columns and rows are orthogonal unit vectors: $$AA^T=I$$ Each entry is a dot product of a column of $A$ with an other column of $A$.
Orthogonal matrices preserve dot products, the length of vectors as well
as the angles between them: $Ax\cdot Ay=x\cdot y$
