# Lagrangian multipliers in complex optimization

I want to know where is my mistake in solving the problem

$$\begin{array}{c} minimize \hspace{1cm} z^*z \\ s.t. \hspace{0.5cm} z = z^* + i \\ \end{array}$$ by using Lagrangian multipliers. $z^*$ is the conjugate of $z$.

It is easy to check that the solution is $z = \frac{1}{2}i.$ However, when I use Lagrangian approach, I cannot find the good solution.

My thoughts:

Let $\mathcal{L} = z^*z + \lambda (z-z^*-i)$ be the Lagrangian function. Thus, we have

$$\frac{\partial \mathcal{L}}{\partial z} = 0 \implies z^*(1-\lambda) = 0,$$ and

$$\frac{\partial \mathcal{L}}{\partial z^*} = 0 \implies z(1+\lambda) = 0 .$$

For any value of $\lambda \in \mathbb{C}$ we have $z =z^* = 0.$

Any help for this problem? Thanks in advance!

This topic has probably long been forgotten, but in the interest of the potential future readers let me just point out the fact that the OP's approach did not fail at all. It seemed to fail for the trivial reason that the derivatives were calculated incorrectly. Using the OP's notation (and the convention of regarding $z$ and $z^*$ as independent variables), we indeed have $$\partial\mathcal{L}/\partial z = z^* + \lambda = 0,$$ $$\partial\mathcal{L}/\partial z^*= z - \lambda = 0.$$ Adding both equations together we find $z+z^* = 0$, and substituting this result back into $z-z^*-\mathrm{i} = 0$, we obtain $2z - \mathrm{i} = 0$, the desired answer.

Let me also note that the expression $\lambda(z - z^* - \mathrm{i})$ is real because it is equal to zero (which is a real number) in the whole domain of the complex plane that we are constrained to by the very definition of the problem.

I'm afraid there are myriad problems with your approach. You're not taking derivatives with respect to complex $z$ correctly; indeed, the fact that you took two, with respect to both $z$ and $z^*$, is suspect---those are the same variable! And you have to be careful building the Lagrangian as well.

The simplest way to do this is to write the equivalent real-valued problem: \begin{array}{ll} \text{minimize} & z_R^2 + z_I^2 \\ \text{subject to} & z_R = z_R + 0 \\ & z_I = - z_I + 1 \\ \end{array} Of course, at this point, we can see the answer by inspection: $z_R=0$, $z_I=1/2$. But if we're crazy enough to build the Lagrangian, we can do this: $$L(z_R,z_I,\lambda_R,\lambda_I) = z_R^2+z_I^2-\lambda_R\cdot 0 - \lambda_I\cdot(2z_I-1)$$ Then the optimality conditions are $$2z_R = 0 \qquad 2z_I - 2\lambda_I = 0 \qquad 2z_I - 1 = 0$$ and again we get $z_R=0$, $z_I=1/2$. $\lambda_R$ is arbitrary, and $\lambda_i = z_I=1/2$.

Now let's show how to do this as in the complex domain. The Lagrange multiplier $\lambda$ is complex, because the equality constraint is complex. But the inner product we need to use is a real inner product $\langle a,b\rangle=\Re(a^*b)$, because the Lagrangian is a real expression: $$L(z,\lambda) = \langle z, z \rangle - \langle \lambda, z-z^*-1 \rangle = z^* z - \Re(\lambda^*(z-z^*-1))$$ So how the heck do you differentiate with respect to complex $z$? Well, I wish I could tell you a good resource here. But what I do is treat it as a multivariate function (because in a real sense, it is), and look at it as a gradient. For linear and quadratic forms, we have $$\nabla_z \langle a, z \rangle = \nabla_z \Re(a^*z) = \nabla_z \Re(az^*) = a, \quad \nabla_z (z^*z) = 2z.$$ So the optimality condition is $$2z - \lambda + \lambda^* = 2z - 2i\Im(\lambda) = 0 \qquad 2z = i$$ and we have $z=\tfrac{1}{2}i$ and $\Im(\lambda)=1/2$, with $\Re(\lambda)$ arbitrary.

I'll be honest, it will not surprise me if someone comes along and criticizes my approach to the calculus in the complex version. I don't think I've ever been given formal instruction, I just derived what is necessary to get the optimization correct. The key here is that it must be mathematically equivalent to the real version if it's going to make any sense. Optimization only occurs with real objectives and real Lagrangians, so the multipliers and inner products and derivatives must be defined to get you there.

• Michael, thanks for your answer. But we can consider $z$ and $z^*$ different variables in complex derivatives. See the Hjorungnes's book: $\textit{Complex-valued Matrix Derivatives}$. This is not a problem.  I have some questions:  (1) Why the Lagrangian should be real?  (2) Could you explain why $\nabla_{z}(z^*z)=2z$? – Alex Silva Mar 10 '15 at 9:58
• I'm sorry, but I am just not buying it. Perhaps you can do that in other contexts but not here. The Lagrangian must be real because it is a quantity to be optimized. How do you take the infimum or supremum of a complex expression? As for 2, again, look at the real and imaginary portions separately and it will be clear. Everything you do in $\mathbb{C}$ must be equivalent to something done in $\mathbb{R}^2$. – Michael Grant Mar 10 '15 at 11:36
• OK, I found a Hjorungnes paper that outlines his proposed approach. I frankly think it's an abuse of notation, for the sake of a perceived convenience that is not really all that convenient. And of course in this case it has tripped you up. It makes no intuitive sense to me that the derivative of $f(z)=z^*$ with respect to $z$ is zero. If I change $z$, $f(z)$ changes, so how can the derivative possibly be zero? – Michael Grant Mar 10 '15 at 12:04
• Regardless... it seems that the primary problem here is that you're necessarily dealing with real-valued expressions, and apparently misapplying his approach in that case. And the paper I saw did not show what to do with real functions anyway. – Michael Grant Mar 10 '15 at 12:07
• Sure! There must be a way to reconcile Hjorungnes approach to this problem, I just don't know what it is. I think what's going on is this. A complex variable has two degrees of freedom. To me, it makes the most sense to choose $\Re z$ and $\Im z$ as the basis. Hjorungnes is suggesting to use $z$ and $z^*$ as the basis. The question that remains is how you use his differentiation approach with real-valued functions. – Michael Grant Mar 10 '15 at 14:07

The answer is easy. You compose first the lagrangian function $$\cal L=z z^* -\lambda(z-z^*-i).$$ Then you have to compute the derivatives and set them to zero: $$\dfrac{\partial \cal L}{\partial z}=z^*-\lambda=0,\\ \dfrac{\partial \cal L}{\partial z^*}=z+\lambda=0,\\ \dfrac{\partial \cal L}{\partial \lambda}=-(z-z^*-i)=0.$$

From the first two equations you have $z-z^*+2\lambda=0.$ Then, comparing to the third equation, you get $\lambda=-i/2,$ and again, from the second equation, you get $z=i/2$ as you wanted.

It might be due to your constraint function $$z=z^*+i$$ is not analytic. So the Lagrange method does not work as expected. I think

• Before posting an answer, I suggest reading the other answers. The reason was already explained by m-j. Moreover, "I think" or "It might be" are not math answers to a question, they are just opinion or guess. By the way, we can extend the concept of derivatives to non-analytic functions (see Wirtinger derivatives). – Alex Silva Aug 18 '20 at 10:05