I have an optimization function in the following form:

$E = \operatorname*{argmin}_{A} \sum_j \| A\bf{x}_j - B \|_2^2 + \mu\sum_i a_{ii}^2$

Where, A is an unknown diagonal matrix with elements $a_{ii}$. $B$ and $\bf{x}$ are known.

My question is how can I differentiate this function to find $A$ ?

I am unclear about how to differentiate $E$ with respect to $A$ as $a_{ii}$ is also occurs in the function.

Thank you.

  • $\begingroup$ Your question is a bit unclear to me- are $a_{ii}$ known? $\endgroup$ – voldemort Sep 3 '14 at 23:27
  • $\begingroup$ No. $a_{ii}$ are the diagonal entries of the unknown diagonal matrix $A$. $\endgroup$ – user570593 Sep 3 '14 at 23:28
  • 1
    $\begingroup$ Are the unknowns the elements of A? In which case, you need to differentiate the function with respect to each element $a_{ij}$ of A, which shouldn't prove to hard when expressing the norm and the matrix vector multiplication as functions of the $a_{ij}$s. $\endgroup$ – Etienne Pellegrini Sep 3 '14 at 23:41

In this case, it's probably easier just to avoid Matrix notation and just consider it a function of $n$ variables. To simplify, let's write $a_i$ rather $a_{ii}$. Also, write $x_{ij}$ for the $i$th component of $x_j$. Then in summation notation, the objective is: $$ \begin{aligned} E &= \sum_j \sum_i (a_{i} x_{ij} - B_i)^2 + \mu \sum_i a_{i}^2\\ \end{aligned} $$ By rearranging the sum, each variable can be minimized independently: $$ \begin{aligned} E &= \sum_i \left(\mu a_i^2 + \sum_j (a_{i} x_{ij} - B_i)^2\right ) \end{aligned} $$ $$ \frac{\partial E}{\partial a_i} = 2 \sum_j (a_{i} x_{ij} - B_i)x_{ij} + 2 \mu a_{i} $$ This gives a minimizer of: $$ a_i^* = \frac{\sum_j B_i x_{ij}} {\mu + \sum_j x_{ij}^2} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.