Closed form solution of $\displaystyle\arg\min_{\alpha \in \Bbb R} \|X - \alpha Y\|_{\text F}^2$ Given $m \times n$ matrices $X$ and $Y$, I am interested in the following least-squares problem.
$$\hat \alpha := \arg\min_{\alpha \in \Bbb R} \|X - \alpha Y\|_{\text F}^2$$
Is there any way to express $\hat \alpha$ in terms of $X$ and $Y$?
 A: Let
$\phi(\alpha)
= \| \mathbf{X} - \alpha \mathbf{Y}\|_F^2
$.
The differential writes
$$
df =
2 (\mathbf{X} - \alpha \mathbf{Y})
:
(- d\alpha \mathbf{Y})
= 
2 \left[ (\alpha \mathbf{Y}-\mathbf{X}):\mathbf{Y}\right]
d\alpha 
$$
The derivative is
$$
\frac{\partial \phi}{\partial \alpha}
=
2(\alpha \mathbf{Y}-\mathbf{X}):\mathbf{Y}
$$
Setting the derivative to zero yields the relation
$
\hat \alpha (\mathbf{Y}:\mathbf{Y}) = \mathbf{X}:\mathbf{Y}
$
from which you can deduce
$$
\hat \alpha
=
\frac{\mathbf{X}:\mathbf{Y}}{\| \mathbf{Y}\|_F^2}
$$
This is analogous to the solution from
CrabMan but in matrix form.
A: In your problem, the matrix structure of $X$ and $Y$ doesn't matter and we can instead interpret them as vectors $x,y \in \mathbb{R}^{mn}$. The optimization problem is
$$\arg \min_{\alpha \in \mathbb{R}} \|x - \alpha y\|.$$
If $y \neq 0$, then the solution of the optimization problem is obviously such $\alpha$ that $\alpha y$ is the orthogonal projection of $x$ onto $y$, i.e.,
$$\alpha y = \left\langle x, \frac{y}{\|y\|} \right\rangle \frac{y}{\|y\|},$$
which gives us
$$\alpha = \frac{\langle x, y \rangle}{\|y\|^2}.$$
