Optimization with Linear constraint $Ax=0$ I confront with this problem:
$$\min_{x \in \mathbb{R}^{n}} \dfrac{1}{2} \left\| x- a \right\|_{2}^{2}$$
subject to $$Ax=0.$$
My tactic is to use Lagrange multiplier method that:
$$\mathcal{L}(x, \lambda) = \dfrac{1}{2} \left\| x- a \right\|_{2}^{2} - \lambda (Ax).$$
Then, $$\nabla_{x} \mathcal{L} = (x-a)-A^{T} \lambda.$$
If I set this $\nabla_{x} \mathcal{L} = 0$, I obtain $x = a + A^{T} \lambda$. However, it seems this is not the solution to the problem because I have not eliminated the $\lambda$ in my solution. Yet, I have to prove this $x$ is the minimizer.
Any helps will much be appreciated.
 A: Let $A \in \mathbb{R}^{m \times n}$. Without loss of generality, we can assume that the rows of $A$ are independent. Thus, $ m \leq n$ (otherwise the problem is infeasible).
The problem is equivalent to the known convex one
$$minimize \hspace{3mm}\|\mathbf{y}\|_{2}^2$$
$$s.t.  \hspace{3mm} A\mathbf{y} = \mathbf{b},$$ where
$\mathbf{y}=\mathbf{x} - \mathbf{a}$, and $\mathbf{b} = -A\mathbf{a}$.
If $m = n$, the solution is $\mathbf{y}^{\star} = A^{-1}\mathbf{b} \Rightarrow \mathbf{x}^{\star} =- A^{-1}A\mathbf{a}+\mathbf{a} \Rightarrow \mathbf{x}^{\star} = \mathbf{0}.$
If $ m < n:$
Lagrangain function: $\mathcal{L} = \|\mathbf{y}\|_{2}^2 + \lambda(A\mathbf{y} -\mathbf{b})$. Thus,
$$\frac{\partial \mathcal{L}}{\partial \mathbf{y}} = 2\mathbf{y}^T+\lambda A = \mathbf{0} \Rightarrow \mathbf{y} = -\frac{1}{2}\lambda A^T \Rightarrow A\mathbf{y} = -\frac{1}{2}\lambda AA^T \Rightarrow $$
$$\mathbf{b} = -\frac{1}{2}\lambda AA^T.$$
Since $rank(A) = m < n$, the matrix $AA^T$ is invertible. Thus, $\lambda = -2(AA^T)^{-1}\mathbf{b}$.
Substituting $\lambda$ in $\mathbf{y}$ expression, we have
$$ \mathbf{y}^{\star} = A^T (AA^T)^{-1}\mathbf{b}.$$
On $\mathbf{x}:$
$$ \mathbf{x}^{\star} = \left[-A^T (AA^T)^{-1}A + I\right]\mathbf{a}.$$
Source: Convex Optimization (S. Boyd and L. Vendenberghe).
