Constrained optimization in random variables I was handed this problem:
Let $X_1,\ldots, X_n$ be a sequence on $n$ independent random variables, with $\mathbb{E}\left[X_i\right]=\mu_i$ and $\mathbb{V}\left[X_i\right]=\sigma_i^2$. Find the constants $a_i$ so that $\mathbb{V}\left[\sum_{i=1}^2 a_i X_i\right]$ is minimized given that $\mathbb{E}\left[\sum_{i=1}^2 a_i X_i\right]=\mu$.
and I came up with the following solution:
It's true that $$\mathbb{E}\left[\sum_{i=1}^{n}a_{i}X_{i}\right]=\sum_{i=1}^{n}a_{i}\mathbb{E}\left[X_{i}\right]=\sum_{i=1}^{n}a_{i}\mu_{i}=\mu$$
and
\begin{align}
\mathbb{V}\left[\sum_{i=1}^{n}a_{i}X_{i}\right]&=\mathbb{E}\left[\left(\left(\sum_{i=1}^{n}a_{i}X_{i}\right)-\mathbb{E}\left[\sum_{i=1}^{n}a_{i}X_{i}\right]\right)^{2}\right]=\mathbb{E}\left[\left(\sum_{i=1}^{n}a_{i}X_{i}-\sum_{i=1}^{n}a_{i}\mu_{i}\right)^{2}\right]\\
&=\mathbb{E}\left[\left(\sum_{i=1}^{n}a_{i}\left(X_{i}-\mu_{i}\right)\right)^{2}\right]=\mathbb{E}\left[\left(\sum_{i=1}^{n}a_{i}\left(X_{i}-\mu_{i}\right)\right)\left(\sum_{i=1}^{n}a_{i}\left(X_{i}-\mu_{i}\right)\right)\right]\\
&=\mathbb{E}\left[\sum_{i=1}^{n}\sum_{j=1}^{n}a_{i}a_{j}\left(X_{i}-\mu_{i}\right)\left(X_{j}-\mu_{j}\right)\right]=\sum_{i=1}^{n}\sum_{j=1}^{n}a_{i}a_{j}\mathbb{E}\left[\left(X_{i}-\mu_{i}\right)\left(X_{j}-\mu_{j}\right)\right]\\
&=\sum_{i=1}^{n}\sum_{j=1}^{n}a_{i}a_{j}Cov\left(X_{i},X_{j}\right).
\end{align}
Since the random variables are independent, for $i\neq j\: : \: Cov\left(X_i,X_j\right)=0$ . So using Kronecker's delta we get
$$\sum_{i=1}^{n}\sum_{j=1}^{n}a_{i}a_{j}Cov\left(X_{i},X_{j}\right)=\sum_{i=1}^{n}\sum_{j=1}^{n}a_{i}a_{j}\delta_{i,j}\mathbb{V}\left[X_{i}\right]=\sum_{i=1}^{n}a_{i}^{2}\sigma_{i}^{2}.$$
So we need to find constants $a_{i}$ so that the following minizes $\sum_{i=1}^{n}a_{i}^{2}\sigma_{i}^{2}$ given that $\sum_{i=1}^{n}a_{i}\mu_{i}=\mu$.
We define the function $L:\mathbb{R}^{n}\to\mathbb{R}$ as
$$L\left(a_{1},a_{2},\ldots,a_{n}\right)=\left(\sum_{i=1}^{n}a_{i}^{2}\sigma_{i}^{2}\right)+\lambda\left(\left(\sum_{i=1}^{n}a_{i}\mu_{i}\right)-\mu\right).$$
The critical points of $L$ are given by
$$\frac{\partial L}{\partial a_{i}}=2\sigma_{i}^{2}a_{i}+\lambda\mu_{i}=0\Rightarrow a_{i}=-\frac{\lambda\mu_{i}}{2\sigma_{i}^{2}},\quad i=1,\ldots,n.$$
Replacing the $a_{i}$ we just found in the constraint we get
$$\sum_{j=1}^{n}-\frac{\lambda\mu_{j}}{2\sigma_{j}^{2}}\mu_{j}=\mu\Rightarrow-\frac{\lambda}{2}\sum_{j=1}^{n}\frac{\mu_{j}^{2}}{\sigma_{j}^{2}}=\mu\Rightarrow\lambda=-\frac{2\mu}{\sum_{j=1}^{n}\frac{\mu_{j}^{2}}{\sigma_{j}^{2}}}.$$
We conclude the the critical point of $L$ are
$$a_{i}=-\left(-\frac{2\mu}{\sum_{j=1}^{n}\frac{\mu_{j}^{2}}{\sigma_{j}^{2}}}\right)\frac{\mu_{i}}{2\sigma_{i}^{2}}=\mu\frac{\mu_{i}}{\sigma_{i}^{2}}\frac{1}{\sum_{j=1}^{n}\frac{\mu_{j}^{2}}{\sigma_{j}^{2}}},\quad i=1,\ldots,n.$$
But now I can't prove that that is a minimum. I thought that I could use the Cauchy - Schwarz inequality, but I'm kinda lost.
PS: Sorry for any and all grammatical errors I'm translating Greek.
Edit. Fix the mistakes pointed out by StubbornAtom.
 A: This is a slight generalization of this question.
By Cauchy-Schwarz,
$$\sum_{i=1}^n (a_i\sigma_i)^2\sum_{i=1}^n \left(\frac{\mu_i}{\sigma_i}\right)^2\ge \left(\sum_{i=1}^n a_i\mu_i\right)^2$$
Or, $$\sum_{i=1}^n a_i^2\sigma_i^2\ge \mu^2\left(\sum\limits_{i=1}^n \frac{\mu_i^2}{\sigma_i^2}\right)^{-1}$$
Equality holds precisely when $a_i\sigma_i$ is proportional to $\frac{\mu_i}{\sigma_i}$ for every $i$, i.e. when $a_i=\frac{c\mu_i}{\sigma_i^2}$ for some non-zero constant $c$. From the constraint $\sum\limits_{i=1}^n a_i\mu_i=\mu$, we have $$c=\mu\left(\sum\limits_{i=1}^n \frac{\mu_i^2}{\sigma_i^2}\right)^{-1}$$
So the optimal solution is $$\hat a_i=\mu\left(\sum\limits_{i=1}^n \frac{\mu_i^2}{\sigma_i^2}\right)^{-1}\frac{\mu_i}{\sigma_i^2}\quad,\,i=1,\ldots,n$$
A: Another possible ad simpler solution is:
We observe that the second partial derivatives of $L$ are
$$\frac{\partial^{2}L}{\partial a_{i}\partial a_{j}}=\begin{cases}
2\sigma_{i}^{2} & i=j\\
0 & i\neq j
\end{cases}.$$
So the Hessian matrix of $L$ will be
$$D^{2}L=\begin{bmatrix}\frac{\partial^{2}L}{\partial a_{1}^{2}} & \frac{\partial^{2}L}{\partial a_{1}\partial a_{2}} & \cdots & \frac{\partial^{2}L}{\partial a_{1}\partial a_{n}}\\
\frac{\partial^{2}L}{\partial a_{2}\partial a_{1}} & \frac{\partial^{2}L}{\partial a_{2}^{2}} & \cdots & \frac{\partial^{2}L}{\partial a_{2}\partial a_{n}}\\
\vdots & \vdots & \ddots & \vdots\\
\frac{\partial^{2}L}{\partial a_{n}\partial a_{1}} & \frac{\partial^{2}L}{\partial a_{n}\partial a_{2}} & \cdots & \frac{\partial^{2}L}{\partial a_{n}^{2}}
\end{bmatrix}=\begin{bmatrix}2\sigma_{1}^{2} & 0 & \cdots & 0\\
0 & 2\sigma_{2}^{2} & \cdots & 0\\
\vdots & \vdots & \ddots & \vdots\\
0 & 0 & \cdots & 2\sigma_{n}^{2}
\end{bmatrix}$$
We see that the eigenvalues of $D^2L$ are $\lambda_{i}=2\sigma_{i}^{2}\geq0,\,i=1,\ldots,n$. That means that $D^2L$ is semi-positive defined matrix, and as a result for $a_{i}=\mu\frac{\mu_{i}}{\sigma_{i}^{2}}\frac{1}{\sum_{j=1}^{n}\frac{\mu_{j}^{2}}{\sigma_{j}^{2}}},\,i=1,\ldots,n$ the value of $\mathbb{V}\left[\sum_{i=1}^{n}a_{i}X_{i}\right]$ is minimized.
