paradoxical derivation involving tall orthonormal matrix I am misunderstanding something very fundamental about linear algebra.
Say A is a tall orthonormal matrix, C is some arbitrary matrix and I want to find X.
$$AX=C$$
Since A is tall orthonormal matrix, $A^TA=I$. Therefore, multiplying $A^T$ from the left, we get
$$X=A^TC$$
But if we plug this back into the first equation,
$$AA^TC \ne C $$
since $AA^T \ne I$.
 A: The issue here is a common misconception about techniques for solving equations, in more than just linear algebra. Your logic shows that
$$AX = C \implies X = A^\top C.$$
This is a logically valid implication. What you are not getting is the converse:
$$X = A^\top C \not\Rightarrow AX = C.$$
Your logic showed that if there is a solution to $AX = C$, then it must be $X = A^\top C$. So, there is at most one solution to the equation. But, it doesn't guarantee that there is a solution. What you are observing is that, for many $C$ (in particular, $C$ such that $\operatorname{colspace}(C) \not\subseteq \operatorname{colspace}(A)$), there is no solution for $X$.
What can we learn from this? Whenever solving an equation of any kind, always check the result in the original equation.
A: Answering my own question... For the first equality to be satisfied, C should have direction in A (The column space of C should be the same as the column space of A). In other words it should have the form $C=AB$ where B is some arbitrary matrix. Plugging this into the last equation,
$AA^TAB=AIB=AB=C$
