# Are those two definitions of orthogonal projection equivalent in a general Hilbert space?

I am taking a graduate level course in probability and we started off with some results in functional analysis.

One thing that I feel I do not understand properly is the definition of an orthogonal projection in the context of Hilbert spaces.

And I was not able to find (sufficiently) exhaustive discussion of the definition (for example not here).

To this end, let a Hilbert space, $H$, possibly infinite dimensional and not necessarily separable. Let a closed convex subset $S \subseteq H$.

1) One the one hand orthogonal projection $P_S :H \to S$ is defined as a mapping of every element $x \in H$ to the unique best approximation of $x$ in $s_0 \in S$. That is $s_0 \in S$ such that $$\|s_0 -x\| = \inf\{s \in S| \|x- s\|\}\,.$$ (Existence of such $P_s$ eventually implies that $H = S\oplus S^{c}$)

2) On the other hand, at various places, I have seen the definition of orthogonal projection to be an operator $P:H \to H$ such that: $P^2 =P$ and $P^* = P$.

$\text{ }$

Here are two things I am trying to figure out:

A. Are those two definitions identical in an (infinite) inseparable case?

To go from 1) to 2) seems to be quite straightforward. However I could not figure out if definition 2) implies 1) in an (infinite) inseparable case.

B. The other thing, if the set $S$ is not closed under addition the operator $P$ might not be linear, so is orthogonal projection required to be linear?

I would appreciate any help.

• separate $H$ into $S = Im(P)$ and $S^{\perp}$, because $H$ is an Hilbert space, for any $u \in S, v \in S^{\perp}$ : $\|u+v\|^2 = \|u\|^2+\|v\|^2$. and $P$ is the identity on $S$ since its restriction to $S$ is an operator $S\to S$ which is invertible and $P^2 = P$ gives $P = Id$. on $S^{\perp}$ clearly $P$ is the zero operator, and it implies that for any $u \in S, v \in S^{\perp}$ : $P(u+v) = u$, hence $Px = \arg\!\min_{ u \in A} \|u-x\|$ – reuns Apr 2 '16 at 0:18
• I think you should ask several questions. – Disintegrating By Parts Apr 2 '16 at 7:17
• Your definition 2 is only for projection onto a closed linear subspace. If the set $S$ is a closed linear subspace, then the two definitions are equivalent. – GEdgar Apr 2 '16 at 15:48
• TrialAndError, yeah maybe question C is not related enough, initially I felt that it might also shed some light on the relation between approximating elements in sets and projection. But now I think that if no one addresses it, maybe it would be best to edit it out of the question. – them Apr 2 '16 at 16:36
• I'm confused why you're specifically asking about (A) in the inseparable case. How does separability help with proving the equivalence? – Eric Wofsey Apr 2 '16 at 19:50

Here's what you can say in general, without topology, completeness or other structure:

Theorem [Projection]: Let $X$ be a real or complex inner product space, and let $M$ be a subspace of $X$. Let $x\in X$ be given. Then $m\in M$ satisfies $$\|x-m\| = \inf_{m'\in M}\|x-m'\|$$ iff $$(x-m) \perp M.$$ If such an $m$ exists, then $m$ is unique.

If $X$ is a real or complex inner product space and $M$ is a linear subspace of $X$, then define $\mathcal{D}_M$ to be the set of $x\in X$ for which there exists $m\in M$ such that $(x-m)\perp M$, and define $P_M : \mathcal{D}_M\subseteq X\rightarrow X$ so that $P_Mx=m$. It is easy to check that $P_M$ is defined for $m\in M$ and $P_M m = m$ because $(m-m)\perp M$. So $M\subseteq \mathcal{D}_M$ always holds.

Theorem [Projection Operator 1]: Let $X$ be a real or complex inner product space, and let $M$ be a linear subspace of $X$. Let $P_M : \mathcal{D}_M \subseteq X\rightarrow X$ be the projection operator described above. Then $\mathcal{D}_M$ is a linear subspace of $X$ containing $M$; $P_M$ is linear on its domain; and $P_M$ has the following properties: \begin{align} \mbox{idempotent:}\;\;\; & P_M^2 x = P_Mx,\;\;\; x\in\mathcal{D}_M,\\ \mbox{symmetric:}\;\;\; & \langle P_M x,y\rangle = \langle x,P_M y\rangle,\;\;\; x,y\in\mathcal{D}_M,\\ \mbox{bounded:}\;\;\; & \|P_M x\| \le \|x\|,\;\;\; x\in\mathcal{D}_M. \end{align}

Suppose $X$ is an inner product space, that $M\subset X$ is a linear subspace of $X$, and suppose that $x \in X$. For any $n =1,2,3,\cdots$, there exists $m_n \in M$ such that $$\|x-m_n\| < \inf_{m'\in M}\|x-m'\| + \frac{1}{n}.$$ It turns out that $\{ m_n \}$ is a Cauchy sequence. Therefore, if $M$ is a complete subspace of $X$ then, then $\lim_n m_n = m$ exists, and you can show by continuity that $\|x-m\|=\inf_{m'\in M}\|x-m'\|$. Assuming $M$ is complete also gives $m\in M$ because $M$ is closed. Hence, $\mathcal{D}_M=X$ in this case, leading to an orthogonal projection $P_M : X\rightarrow X$.

A notable example where $\mathcal{D}_M= X$ is where $M$ is finite-dimensional. This follows because $M$ is complete, but you can also demonstrate this directly by choosing an orthonormal basis $\{e_n\}_{n=1}^{N}$ of $M$ and verifying that $$\left(x - \sum_{n=1}^{N}\langle x,e_n\rangle x_n\right)\perp M.$$ So the orthogonal projection onto $M$ is $P_M = \sum_{n=1}^{N}\langle x,e_n\rangle e_n$.

If $M$ is a closed subspace of a Hilbert space $H$, then $M$ is complete and, hence, $\mathcal{D}_M=H$. In this case, $P_M$ is defined on all of $H$, is linear, and satisfies $P_M=P_M^2=P_M^{\star}$.

EXAMPLE: Let $X=L^2[0,1]$. Let $M_r = \{\chi_{[0,r]}f : f \in L^2[0,1]\}$. For each $f \in X$, it is easy to check that $$(f - \chi_{[0,r]}f) \perp M_r.$$ Therefore $P_{M_r}f = \chi_{[0,r]}f$ and, without checking, $P_{M_r}$ must be linear and satisfy $$P_{M_r}^{2} = P_{M_r} = P_{M_r}^{\star},\\ \|P_{M_r}f\| \le \|f\|,\;\;\; f\in L^2[0,1].$$

Theorem [Projection Operator 2]: Let $X$ be a real or complex inner product space and let $P$ be a linear operator on $X$ such that $P=P^2$ and $P$ is symmetric. Then $P$ is the orthogonal projection onto $\mathcal{R}(P)$.

Proof: For $P$ as stated, let $M=P(X)$ be the range of $P$. Then $M$ is a subspace, and, for every $x\in X$, one has $$(x-Px)\perp M$$ because $\langle x-Px,Py\rangle = \langle Px-P^2x,y\rangle= \langle Px-Px,y\rangle=0$ for all $y\in X$. Therefore $P$ is the orthgonal projection onto $M$. $\;\;\;\blacksquare$.

Although projections on closed convex subsets $S$ are sometimes considered, the term "orthogonal projection" generally refers to a linear operator, which is the case where $S$ is a closed linear subspace (your definition (2)). The subspace in question is the range of the operator $P$, which is closed e.g. because it is the null space of $I - P$.

• Thanks, I think it answers A and B, so regarding A: the two definitions are equivalent. Since one can then show (assuming my def. 2) that for $x \in H$, $Px$ would be the best approximation in the range of $P$ (just using Pythagoras: $s \in range(P)$, $\|x-Px - s\| = \|x-Px\| + \|s\|$). – them Apr 1 '16 at 23:34