# how to differentiate to optimize this function?

I have an optimization function in the following form:

$E = \operatorname*{argmin}_{A} \sum_j \| A\bf{x}_j - B \|_2^2 + \mu\sum_i a_{ii}^2$

Where, A is an unknown diagonal matrix with elements $a_{ii}$. $B$ and $\bf{x}$ are known.

My question is how can I differentiate this function to find $A$ ?

I am unclear about how to differentiate $E$ with respect to $A$ as $a_{ii}$ is also occurs in the function.

Thank you.

• Your question is a bit unclear to me- are $a_{ii}$ known? – voldemort Sep 3 '14 at 23:27
• No. $a_{ii}$ are the diagonal entries of the unknown diagonal matrix $A$. – user570593 Sep 3 '14 at 23:28
• Are the unknowns the elements of A? In which case, you need to differentiate the function with respect to each element $a_{ij}$ of A, which shouldn't prove to hard when expressing the norm and the matrix vector multiplication as functions of the $a_{ij}$s. – Etienne Pellegrini Sep 3 '14 at 23:41

In this case, it's probably easier just to avoid Matrix notation and just consider it a function of $n$ variables. To simplify, let's write $a_i$ rather $a_{ii}$. Also, write $x_{ij}$ for the $i$th component of $x_j$. Then in summation notation, the objective is: \begin{aligned} E &= \sum_j \sum_i (a_{i} x_{ij} - B_i)^2 + \mu \sum_i a_{i}^2\\ \end{aligned} By rearranging the sum, each variable can be minimized independently: \begin{aligned} E &= \sum_i \left(\mu a_i^2 + \sum_j (a_{i} x_{ij} - B_i)^2\right ) \end{aligned} $$\frac{\partial E}{\partial a_i} = 2 \sum_j (a_{i} x_{ij} - B_i)x_{ij} + 2 \mu a_{i}$$ This gives a minimizer of: $$a_i^* = \frac{\sum_j B_i x_{ij}} {\mu + \sum_j x_{ij}^2}$$