# How to run gradient descent algorithm when parameter space is constrained?

I would like to minimize a function with one constraint.

Suppose I have two variables x and y, their product is a convergent function, so the first cost function is J1 = (xy), and other two variables z and t, their product is convergent function. so the second cost function is J2 = (zt).

I already used the gradient descent to optimize squared error between the two functions, So the new cost function was Jt = (|J1 - J2|.^2).

My question is, in my cost functions, I am always interested in optimizing the minimal result of the product, but we don't care if variable x = z. it means if x - z = 0. So, I need to add a constraint to optimize my cost function Jt with a condition of x - z = 0.

Does Barrier methods or The Lagrangian will make sense if I used them? What are some good (simple) algorithms that deal with this constrained optimization problem? I'm hoping for just a simple fix to my algorithm.

thank you so much.

• You cannot run gradient descent, but you can run the projected gradient method. However, it seems your function is nonconvex, so you are not guaranteed to find the optimum. – Alex Shtof May 8 '18 at 10:49
• @Alex Yes, for that I need to add a constraint x - z = 0 which is a convex function. I mean I want to minimize my cost function till x - z = 0 which is the appropriate limit of optimization of my cost function. could you please explain how to use projected gradient method in that function? thanks – Zeyad_Zeyad May 8 '18 at 15:20
• I cannot. But I believe that your favorite search engine can. Just click the first search result for "projected gradient" – Alex Shtof May 8 '18 at 16:43
• @Alex ,, Yes I know that I should read about it, but my question, did you get the idea i want to make? is that feasible? I mean is that possible to be done? – Zeyad_Zeyad May 9 '18 at 10:24