Simple proof that $\min \|\mathbf{f} - \sum_i \lambda_i\hat{\mathbf{g}_i}\|$ is given by $\lambda_i = \langle \mathbf{f}, \hat{\mathbf{g}}_i \rangle$ Let $\mathbf{f}$ be a real valued vector and $\{\hat{\mathbf{g}}_i\}$ be a set of orthonormal vectors of the same dimension. What is a simple proof that
$$\min \|\mathbf{f} - \sum_i \lambda_i\hat{\mathbf{g}_i}\|$$
is given by $\lambda_i = \langle \mathbf{f}, \hat{\mathbf{g}}_i \rangle$
This is my attempt:
$$\min \|\mathbf{f} - \sum_i \lambda_i\hat{\mathbf{g}}_i\|$$
$$ = \min \|\mathbf{f}\|^2 + \|\sum_i \lambda_i\hat{\mathbf{g}}_i\|^2 - 2\langle\mathbf{f},\sum_i \lambda_i\hat{\mathbf{g}}_i\rangle$$
Since 
$$\|\sum_i \lambda_i\hat{\mathbf{g}}_i\|^2 = \langle\sum_i\lambda_i\hat{\mathbf{g}}_i,\sum_i \lambda_i\hat{\mathbf{g}}_i\rangle$$
$$ = \sum_i\sum_j\langle\lambda_i\hat{\mathbf{g}}_i, \lambda_j\hat{\mathbf{g}}_j\rangle$$
$$ = \sum_i\langle\lambda_i\hat{\mathbf{g}}_i, \lambda_i\hat{\mathbf{g}}_i\rangle$$
$$ = \sum_i\lambda_i^2$$
Thus 
$$\min \|\mathbf{f} - \sum_i \lambda_i\hat{\mathbf{g}_i}\| = \min \sum_i \lambda_i^2 - 2\langle\mathbf{f},\sum_i \lambda_i\hat{\mathbf{g}}_i\rangle$$
$$ = \min \sum_i \lambda_i^2 - 2 \sum_i \lambda_i\langle\mathbf{f},\hat{\mathbf{g}}_i\rangle$$
Since at the minimum we have
$$\frac{d}{d\boldsymbol{\lambda}} \sum_i \lambda_i^2 - 2 \sum_i \lambda_i\langle\mathbf{f},\hat{\mathbf{g}}_i\rangle = 0$$
Taking partial derivative w.r.t. $\lambda_i$ we get
$$2 \lambda_i - 2 \langle\mathbf{f},\hat{\mathbf{g}}_i\rangle = 0$$
$$\lambda_i = \langle\mathbf{f},\hat{\mathbf{g}}_i\rangle$$
but is there something simpler?
 A: Hint: An arbitrary $\mathbf{f}$ can be expressed uniquely as $\mathbf{f} = \sum_i \mu_i \mathbf{\hat{g}}_i + \mathbf{f}'$ where $\mathbf{f}'$ is orthogonal to all the $\mathbf{\hat{g}}_i$.

Second hint: $$\lVert \mathbf{f} - \sum_i \lambda_i \hat{\mathbf{g}}_i \rVert^2 = \lVert \mathbf{f}'\rVert^2 + \sum_i \lVert (\mu_i - \lambda_i)\hat{\mathbf{g}}_i\rVert^2$$

Summary of a solution: Using the above two hints, we find that
$$\lVert \mathbf{f} - \sum_i \lambda_i \hat{\mathbf{g}}_i  \rVert^2 =  \lVert \mathbf{f}'\rVert^2 + \sum_i |\mu_i - \lambda_i|^2$$
This is a sum of a constant and a series of positive numbers, so it is minimized if we can make those positive numbers zero. We can: $\lambda_i = \mu_i$. But looking back at the first hint, $\mu_i = \langle \hat{\mathbf{g}}_i, \mathbf{f} \rangle$ so we're done.
A: As a function of $\lambda_i$, the norm is convex in each $\lambda_i$. Picking $\lambda_i=\langle f,g_i\rangle$ gives a value of 0 for the norm. Since norms are non-negative, you're done.
