I am taking a graduate level course in probability and we started off with some results in functional analysis.

One thing that I feel I do not understand properly is the definition of an orthogonal projection in the context of Hilbert spaces.

And I was not able to find (sufficiently) exhaustive discussion of the definition (for example not here).

To this end, let a Hilbert space, $H$, possibly infinite dimensional and not necessarily separable. Let a closed convex subset $S \subseteq H$.

1) One the one hand orthogonal projection $P_S :H \to S$ is defined as a mapping of every element $x \in H$ to the unique best approximation of $x$ in $s_0 \in S$. That is $s_0 \in S$ such that $$\|s_0 -x\| = \inf\{s \in S| \|x- s\|\}\,. $$ (Existence of such $P_s$ eventually implies that $H = S\oplus S^{c}$)

2) On the other hand, at various places, I have seen the definition of orthogonal projection to be an operator $P:H \to H$ such that: $P^2 =P$ and $P^* = P$.

$\text{ }$

Here are two things I am trying to figure out:

A. Are those two definitions identical in an (infinite) inseparable case?

To go from 1) to 2) seems to be quite straightforward. However I could not figure out if definition 2) implies 1) in an (infinite) inseparable case.

B. The other thing, if the set $S$ is not closed under addition the operator $P$ might not be linear, so is orthogonal projection required to be linear?

I would appreciate any help.

  • $\begingroup$ separate $H$ into $S = Im(P)$ and $S^{\perp}$, because $H$ is an Hilbert space, for any $u \in S, v \in S^{\perp}$ : $\|u+v\|^2 = \|u\|^2+\|v\|^2$. and $P$ is the identity on $S$ since its restriction to $S$ is an operator $S\to S$ which is invertible and $P^2 = P$ gives $P = Id$. on $S^{\perp}$ clearly $P$ is the zero operator, and it implies that for any $u \in S, v \in S^{\perp}$ : $P(u+v) = u$, hence $Px = \arg\!\min_{ u \in A} \|u-x\| $ $\endgroup$ – reuns Apr 2 '16 at 0:18
  • $\begingroup$ I think you should ask several questions. $\endgroup$ – Disintegrating By Parts Apr 2 '16 at 7:17
  • $\begingroup$ Your definition 2 is only for projection onto a closed linear subspace. If the set $S$ is a closed linear subspace, then the two definitions are equivalent. $\endgroup$ – GEdgar Apr 2 '16 at 15:48
  • $\begingroup$ TrialAndError, yeah maybe question C is not related enough, initially I felt that it might also shed some light on the relation between approximating elements in sets and projection. But now I think that if no one addresses it, maybe it would be best to edit it out of the question. $\endgroup$ – them Apr 2 '16 at 16:36
  • $\begingroup$ I'm confused why you're specifically asking about (A) in the inseparable case. How does separability help with proving the equivalence? $\endgroup$ – Eric Wofsey Apr 2 '16 at 19:50

Here's what you can say in general, without topology, completeness or other structure:

Theorem [Projection]: Let $X$ be a real or complex inner product space, and let $M$ be a subspace of $X$. Let $x\in X$ be given. Then $m\in M$ satisfies $$ \|x-m\| = \inf_{m'\in M}\|x-m'\| $$ iff $$ (x-m) \perp M. $$ If such an $m$ exists, then $m$ is unique.

If $X$ is a real or complex inner product space and $M$ is a linear subspace of $X$, then define $\mathcal{D}_M$ to be the set of $x\in X$ for which there exists $m\in M$ such that $(x-m)\perp M$, and define $P_M : \mathcal{D}_M\subseteq X\rightarrow X$ so that $P_Mx=m$. It is easy to check that $P_M$ is defined for $m\in M$ and $P_M m = m$ because $(m-m)\perp M$. So $M\subseteq \mathcal{D}_M$ always holds.

Theorem [Projection Operator 1]: Let $X$ be a real or complex inner product space, and let $M$ be a linear subspace of $X$. Let $P_M : \mathcal{D}_M \subseteq X\rightarrow X$ be the projection operator described above. Then $\mathcal{D}_M$ is a linear subspace of $X$ containing $M$; $P_M$ is linear on its domain; and $P_M$ has the following properties: \begin{align} \mbox{idempotent:}\;\;\; & P_M^2 x = P_Mx,\;\;\; x\in\mathcal{D}_M,\\ \mbox{symmetric:}\;\;\; & \langle P_M x,y\rangle = \langle x,P_M y\rangle,\;\;\; x,y\in\mathcal{D}_M,\\ \mbox{bounded:}\;\;\; & \|P_M x\| \le \|x\|,\;\;\; x\in\mathcal{D}_M. \end{align}

Suppose $X$ is an inner product space, that $M\subset X$ is a linear subspace of $X$, and suppose that $x \in X$. For any $n =1,2,3,\cdots$, there exists $m_n \in M$ such that $$ \|x-m_n\| < \inf_{m'\in M}\|x-m'\| + \frac{1}{n}. $$ It turns out that $\{ m_n \}$ is a Cauchy sequence. Therefore, if $M$ is a complete subspace of $X$ then, then $\lim_n m_n = m$ exists, and you can show by continuity that $\|x-m\|=\inf_{m'\in M}\|x-m'\|$. Assuming $M$ is complete also gives $m\in M$ because $M$ is closed. Hence, $\mathcal{D}_M=X$ in this case, leading to an orthogonal projection $P_M : X\rightarrow X$.

A notable example where $\mathcal{D}_M= X$ is where $M$ is finite-dimensional. This follows because $M$ is complete, but you can also demonstrate this directly by choosing an orthonormal basis $\{e_n\}_{n=1}^{N}$ of $M$ and verifying that $$ \left(x - \sum_{n=1}^{N}\langle x,e_n\rangle x_n\right)\perp M. $$ So the orthogonal projection onto $M$ is $P_M = \sum_{n=1}^{N}\langle x,e_n\rangle e_n$.

If $M$ is a closed subspace of a Hilbert space $H$, then $M$ is complete and, hence, $\mathcal{D}_M=H$. In this case, $P_M$ is defined on all of $H$, is linear, and satisfies $P_M=P_M^2=P_M^{\star}$.

EXAMPLE: Let $X=L^2[0,1]$. Let $M_r = \{\chi_{[0,r]}f : f \in L^2[0,1]\}$. For each $f \in X$, it is easy to check that $$ (f - \chi_{[0,r]}f) \perp M_r. $$ Therefore $P_{M_r}f = \chi_{[0,r]}f$ and, without checking, $P_{M_r}$ must be linear and satisfy $$ P_{M_r}^{2} = P_{M_r} = P_{M_r}^{\star},\\ \|P_{M_r}f\| \le \|f\|,\;\;\; f\in L^2[0,1]. $$

Theorem [Projection Operator 2]: Let $X$ be a real or complex inner product space and let $P$ be a linear operator on $X$ such that $P=P^2$ and $P$ is symmetric. Then $P$ is the orthogonal projection onto $\mathcal{R}(P)$.

Proof: For $P$ as stated, let $M=P(X)$ be the range of $P$. Then $M$ is a subspace, and, for every $x\in X$, one has $$ (x-Px)\perp M $$ because $\langle x-Px,Py\rangle = \langle Px-P^2x,y\rangle= \langle Px-Px,y\rangle=0$ for all $y\in X$. Therefore $P$ is the orthgonal projection onto $M$. $\;\;\;\blacksquare$.


Although projections on closed convex subsets $S$ are sometimes considered, the term "orthogonal projection" generally refers to a linear operator, which is the case where $S$ is a closed linear subspace (your definition (2)). The subspace in question is the range of the operator $P$, which is closed e.g. because it is the null space of $I - P$.

  • $\begingroup$ Thanks, I think it answers A and B, so regarding A: the two definitions are equivalent. Since one can then show (assuming my def. 2) that for $x \in H$, $Px$ would be the best approximation in the range of $P$ (just using Pythagoras: $s \in range(P)$, $\|x-Px - s\| = \|x-Px\| + \|s\|$). $\endgroup$ – them Apr 1 '16 at 23:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.