# Using gradient descent and Newton's method combined

I have this function $f(\mathrm{X})$ where $\mathrm{X=A+B+C}$ where $\mathrm{A}$ is a diagonal element with variable $a$ on its diagonal. $\mathrm{B}$ is another diagonal matrix with variable $b$ on its diagonal. $\mathrm{C}$ is a positive definite matrix with variable $c_{ij}$. Now I want to optimize this function over these variables. I was wondering to keep $\mathrm{A}$ and $\mathrm{B}$ constant first optimize over $\mathrm{C}$ using Newton's method. Then I will keep $\mathrm{C}$ constant and optimize over $a$ and $b$ using gradient descent.
I am not sure if this is applicable or will work. Suggestions guys?

-
What is motivating you to use two different methods? –  Tpofofn Aug 26 '13 at 1:34
@Tpofofn. I have Newton's method implementation that guarantees positive definiteness. And gradient descent over the other variable such that they are greater than equal to 0 will guarantee positive definiteness. I am not sure how I will guarantee positive definiteness if I optimize over $c_{ij}$ –  user34790 Aug 26 '13 at 2:05
It sounds like you should formulate your problem as a semidefinite program, then solve it using a method tailored for SDPs, like an interior point method. Good software is available, such as CVX in Matlab. –  littleO Aug 26 '13 at 6:19
@littleO. I have log(det(X)) term in the function. I am not sure if this satisfied the criteria for semidefinite programming? –  user34790 Aug 26 '13 at 17:18
Maybe not exactly a semidefinite program, but I still think there's a good chance that an interior point method would be the standard way to solve your problem, and perhaps it could be done without too much trouble in CVX. $\log(\det X)$ is a function often encountered in convex optimization. Can you write out your optimization problem explicitly? –  littleO Aug 26 '13 at 22:07

Here's some sample Matlab code that uses CVX to solve a problem similar to yours. However, I don't yet know how to handle the $r^T X^{-1} r$ term.

N = 30;
y = randn(N,1);

cvx_begin sdp

variable X hermitian
minimize(.5*y'*X*y - log_det(X))
subject to
X >= 0

cvx_end

-
I guess I cannot use $X^{-1}$ with cvx, is that so? –  user34790 Aug 28 '13 at 15:02
@littlO I used your example it doesn't work. It shows failed status –  user34790 Aug 29 '13 at 10:42