Is there a solution to $Tx=y$ such that $\|x\|$ is minimal? I recently used the method of least norms to solve an underdetermined system of linear equations for a problem at work.  This got me thinking, if I were to think about this more generally, does such a solution exits?
That is, if $T$ is an operator on an inner-product space $V$, and if $x,y\in V$, is it possible to find a solution to 
$$Tx=y$$
such that $\|x\|$ is minimal?
I know that this doesn't exactly line up with the problem at work because there I was going from $\mathbb{R}^n$ to $\mathbb{R}^m$, but I thought it may be simpler to consider the case of an operator on an inner product space than a linear map between two different spaces.
I want to write this up to present it to a class, so if you know of any good sources where I might read up on this in depth, I would appreciate that as well.
 A: For a solution to exist we must require that $y$ is in the range of $T$.
Next we note that adding an element of the null space $\mathcal{N}$ of $T$
to $x$ does not change things. Thus the allowable $x$ are
\begin{equation*}
x=u+v,\;u\perp \mathcal{N},\;v\in \mathcal{N}
\end{equation*}
Note that $u$ is unique since if $u_{1}$ and $u_{2}$ are both $\perp
\mathcal{N}$ then their difference is in $\mathcal{N}$. Now
\begin{equation*}
\parallel x\parallel ^{2}=\parallel u\parallel ^{2}+\parallel v\parallel ^{2}
\end{equation*}
which is minimal for $v=0$.
A: The Moore-Penrose generalised inverse of T applied to y gives the least norm vector x that minimises ||Tx-y||. Note that every linear map has such an inverse, even if the dimensions of the domain and codomain are different.
The usual way to compute such and inverse is via the SVD.
A: In general inner product space it is not true that there exists such a $x_0$ such that $Tx_0=y$ and $||x_0||\le||x||$ for all $x$ that satisfies $Tx=y$. Consider the inner product space $V=l^2$ where a linear operator $T$ is defined by $$Te_k=k\cdot e_1 $$
where $e_k=(\delta_{nk})_{n=1}^{\infty}$. It is not hard to see that $x=\frac1k e_k$ is a solution to $Tx=e_1$ for all $k\in\Bbb N$ while $||x||$ can be made arbitrary small.
However, in the case where $V$ is finite dimensional is different. Suppose that $Tx=y$ has a solution, then $S=T^{-1}\{y\}$ is a non-empty closed subset of $V$ since $T$ is continuous. For any $x_1,x_2\in S$ and $0\le t\le 1$ we have 
$$T(tx_1+(1-t)x_2)=tTx_1+(1-t)Tx_2=ty+(1-t)y=y
$$
so $S$ is a convex set. Since a $V$ is a Hilbert space (because $\dim V<\infty$) and $S\subset V$ is a closed and convex subset of $X$, $S$ contains an element with minimum norm. Therefore there exist $x_0\in S$ such that $Tx_0=y$ and $||x_0||$ has the smallest norm possible.
The case where $T:V\to W$ can be handle similarly provided that $V$ is finite dimensional and $Tx=y$ has at least one solution.
