# minimizing linear combination of inner products

$\mathbf{x},\mathbf{y_1},\mathbf{y_2}\in \mathbb{R}^{m}$ and $\alpha_1,\alpha_2 \in \mathbb{R}$. Also $\|\mathbf{y_1}\|_2 = \|\mathbf{y_2}\|_2 = 1$ and $\alpha_1\geq\alpha_2\geq0$.

How should we find $\mathbf{x}$ that minimizes

$$\begin{array}{c} \text{min} \hspace{4mm} \alpha_1 \langle \mathbf{x},\mathbf{y_1} \rangle + \alpha_2 \langle \mathbf{x},\mathbf{y_2} \rangle \\ s.t \hspace{17mm}\|\mathbf{x}\|_2 = 1, \end{array}$$

$\mathbf{y_1},\mathbf{y_2}$ are pointing in arbitrary directions.
We can start by considering $\alpha_1=\alpha_2=1$ and find $\mathbf{x}$. But how to approach when $\alpha_1\neq\alpha_2$ is the problem.

Variant of original formulation (added later)

$$\begin{array}{c} \text{min} \hspace{4mm} \alpha_1 \big(\langle \mathbf{x},\mathbf{y_1} \rangle \big)^2+ \alpha_2 \big(\langle \mathbf{x},\mathbf{y_2} \rangle \big)^2 \\ s.t \hspace{17mm}\|\mathbf{x}\|_2 = 1, \end{array}$$

• Yes only variable is $x$. and I removed my "interesting problem" comment from original post. You are right…its trivial. how about the second problem that I added later – NAASI Jan 28 '15 at 19:07

Since $\alpha_1\langle x,y_1\rangle+\alpha_2\langle x,y_2\rangle=\langle x,y\rangle$ with $y:=\alpha_1y_1+\alpha_2y_2$, the problem is equivalent to finding $x$ such that $\langle x,y\rangle$ is minimal with $\|x\|_2=1$.

The Cauchy-Schwarz inequality states that $|\langle x,y\rangle|\leq\|x\|_2\|y\|_2=\|y\|_2$ with the equality if and only if $x$ is a scalar multiple of $y$. Varying $x$ over the boudnary of the unit ball, $\langle x,y\rangle$ hence attains the values between $-\|y\|_2$ and $\|y\|_2$, with the minimum achieved by $x=x_*:=-y/\|y\|_2$.

EDIT For the second problem, since $\alpha$'s are nonnegative, you can "merge" them into $y$'s and consider a problem of minimizing $\langle x,z_1\rangle^2+\langle x,z_2\rangle^2$ instead. Note that with $Z:=[z_1,z_2]$, we have $$\langle x,z_1\rangle^2+\langle x,z_2\rangle^2=\|Z^Tx\|_2^2.$$ If $Z=USV^T$ is the SVD of $Z$ ($U$ is $m\times 2$ with orthonormal columns, $V$ is $2\times 2$ orthogonal matrix, and $S$ is $2\times 2$ diagonal), the minimizer $x_*$ is hence given by the left singular vector associated with the minimal singular value of $Z$. The minimal value is given by the square of this minimal singular value.

• I was thinking of using $\alpha_1\langle x,y_1\rangle^2 + \alpha_2\langle x,y_2\rangle^2=\alpha_1x^Ty_1y_1^Tx + \alpha_2x^Ty_2y_2^Tx = x^T \big(\alpha_1y_1y_1^T + \alpha_2^Ty_2y_2^T \big)x$ and this matches to your formulation but here we didnt change the dimension as was in the case of $Z$. Your reply was very helpful. Thank you – NAASI Jan 29 '15 at 17:36