is $U^{2}=P$? where $U$ unitary and $P$ orthogonal projection

Problem:

I am trying to solve the following problem, but I couldn't. The problem is:

Let $U$ be unitary matrix. Let $P$ and $UP$ be orthogonal projections. Is it true that $U^{2}=P$? If yes, please show me how to prove it.

-
What if $P=0$? Or what if the nullspace of $P$ has dimension at least $2$, and $U$ exchanges two orthonormal vectors in a given basis for $N(P)$? – Arturo Magidin Apr 24 '12 at 3:49

A very strange question. $U^2$ is also a unitary matrix, so it can't be $P$ unless $P = I$ (in which case $UP=U$ must also be $I$, because that's the only orthogonal projection that is unitary). For any other orthogonal projection $P$, you can get counterexamples by taking $U$ to be a unitary matrix for which $\text{Ran}(P)$ and $\text{Ker}(P)$ are invariant subspaces, with $Ux=x$ for $x \in \text{Ran}(P)$.

-

Note that if $P=U^2$, then $P$ would be invertible; but since $P^2=P$ (being a projection) that would imply that $P=I$.

So the answer would be "yes" if and only if the only way for $P$ and $UP$ to both be orthogonal projections when $U$ is unitary would be for $P$ to be the identity.

This is never true, since we can always take $U=I$ no matter what $P$ is. And we can always take $P=0$, no matter what $U$ is. In neither case will it follow necessarily that $U^2=P$.

So let us make this a bit more interesting and add the assumption that $U\neq I$ and that $P\neq 0$. Will the implication hold then?

If $\dim(V)\geq 3$, this is not the case: let $P$ be an orthogonal projection with nullspace of dimension $2$. Let $\mathbf{e}_1,\mathbf{e}_2,\mathbf{e}_3,\ldots,\mathbf{e}_n$ be an orthonormal basis, where $\mathbf{e}_1,\mathbf{e}_2$ are basis for the nullspace, and $\mathbf{e}_3,\ldots,\mathbf{e}_n$ are a basis for the range. Then let $U$ be the linear transformation that exchanges $\mathbf{e}_1$ and $\mathbf{e}_2$, and leaves the rest of the vectors invariant. Then $UP=P$, but $P\neq U^2$, since $P$ is not invertible.

If $\dim(V)= 2$, then the result is still false: if $P\neq 0$, then $P$ is either similar to $P(x,y) = (x,0)$, or is the identity. If $P$ is (essentially) $P(x,y)=(x,0)$, then $UP$ must send $(0,y)$ to $(0,0)$ hence $\{(0,y)\}^{\perp} = \{(x,0)\}$ to itself; that is, $U$ must send $(1,0)$ to $(1,0)$, and must send $(0,1)$ to $(0,u)$ where $u$ is a unit of the underlying field. Either way, $U^2\neq P$.

If $\dim(V)=1$, then the only nonzero projection is the identity; the result will hold since $U$ must act like the identity on the image of $P$, hence if $P\neq 0$ then $U=P=I$.

-