3
$\begingroup$

I'm using TensorFlow for some computations with complex variables (and derivatives of these computations). When I compute the derivative of (simple) holomorphic functions, the results obtained with TensorFlow are the conjugate of what I would expect. A simple example:

Given $z = x + yi$, and $f(z) = zz = x^2 + i2xy - y^2$.

We have $\frac{df}{dz} = \frac12 \left(\frac{\partial f}{\partial x} - i\frac{\partial f}{\partial y}\right) = 2x + i2y$.

Hence, for $z = \frac{1}{5}i$, $\frac{df}{dz} = \frac{2}{5}i$. However, using TensorFlow, I obtain $\frac{df}{dz} = -\frac{2}{5}i$. While searching for this, I found the following statement: "The gradient of a holomorphic function is the conjugate of its complex derivative." in a Github Issue, however, I don't understand why. In other words: Why is the gradient of a holomorphic functions equal to the conjugate of the complex derivative, and, where is the mistake in this simple example?

$\endgroup$
1
  • $\begingroup$ $f(z)=z^2\rightarrow f'(z)=2z\rightarrow f'\left(\frac{1}{5}\,i\right)=\frac{2}{5}i$. Period. $\endgroup$
    – Raffaele
    Jul 8, 2017 at 16:03

3 Answers 3

3
$\begingroup$

Couple of years late, but I came across this issue too and did some digging.

The key point is that TensorFlow defines the "gradient" of a complex-valued function $f(z)$ of a complex variable as "the gradient of the real map $F: (x,y)\mapsto Re({f(x+iy)})$, expressed as a complex number" (the gradient of that real map is a vector in $\mathbb R^2$, so we can express it as a complex number in the obvious way).

Presumably the reason for that definition is that in TF one is usually concerned with gradients for the purpose of running gradient descent on a loss function, and in particular for identifying the direction of maximum increase/decrease of that loss function. Using the above definition of gradient means that a complex-valued function of complex variables can be used as a loss function in a standard gradient descent algorithm, and the result will be that the real part of the function gets minimised (which seems to me a somewhat reasonable interpretation of "optimise this complex-valued function").

Note that an equivalent way to write that definition of gradient is $$ gradient(f) := \frac{dF}{dx} + i\frac{dF}{dy}=\overline{\frac{df}{dz} + \frac{d\overline{f}}{dz}},$$ which for a holomorphic function simply reduces to $\overline{\frac{df}{dz}}$.

I wrote up a longer explanation on the GitHub issue you linked: https://github.com/tensorflow/tensorflow/issues/3348#issuecomment-512101921

$\endgroup$
2
  • $\begingroup$ Welcome to Mathematics Stack Exchange! A quick tour will enhance your experience. Here are helpful tips to write a good question and write a good answer. For equations, please use MathJax. $\endgroup$
    – dantopa
    Jul 18, 2019 at 0:48
  • $\begingroup$ I've tried and come to realize what you said above. The execution of tensorflow is for real-value functions. Thus, any complex-valued functions should be rewrite in terms of a sum of 2 real-valued functions, then compute the gradients for each of these two functions. $\endgroup$
    – pej
    Nov 20, 2020 at 12:57
2
$\begingroup$

A holomorphic function $z\mapsto f(z)$ has no gradient, but at each point $z$ in its domain $\Omega$ a derivative $f'(z)\in{\mathbb C}$. In this way to $f$ a new function $f': \> z\mapsto f'(z)$ is associated, and it turns out that $f'$ is again holomorphic in $\Omega$.

Now you can write $f(x+iy)=u(x,y)+i v(x,y)$ with real-valued functions $u$ and $v$. In this way $f$ can be viewed as a vector-valued function $${\bf f}:\quad\Omega\to{\mathbb R}^2, \qquad (x,y)\mapsto\bigl(u(x,y),v(x,y)\bigr)\ .$$ This ${\bf f}$ does not have a gradient either, but at each point $(x,y)\in\Omega$ a Jacobian (or derivative) $d{\bf f}(x,y)$ whose matrix has the special form $$\left[\matrix{A&-B\cr B&A\cr}\right]\ ,$$ whereby the numbers $A$ and $B$ are related to the derivative of the holomorphic function $f$ we started with by $f'(x+iy)=A+iB$. This is the content of the so-called CR-equations.

$\endgroup$
2
  • $\begingroup$ Thanks a lot. My math isn't that good, but I've been reading up based on this answer and it helped a lot. One follow up on the last sentences. TensorFlow seems to use the Jacobian during optimization (resulting in $-\frac{2}{5}i$ in my question). So, is it correct that for optimization I can treat the functions as ${\bf f}: \Omega \to \mathbb{R}^2$ and use the aforementioned Jacobian? $\endgroup$ Jul 11, 2017 at 7:50
  • 1
    $\begingroup$ I don't know what TensorFlow does; you would have to check the manual. In any case, for the rest of the world, when $f(z)=z^2$ then $f'(z)=2z$, and the Jacobian of the associated ${\bf f}$ is the matrix $$\left[\matrix{2x &-2y\cr 2y&2x\cr}\right]\ .$$ $\endgroup$ Jul 11, 2017 at 10:10
1
$\begingroup$

It is late but I am writing for anyone encountering the same problem as yours. The explanation focuses on why tf.gradients in Tensorflow yeilds (-2j/5) but not (2j/5). First, tf.gradients only handles real-valued functions, thus it converts your original function $f(z)=x^2+2jxy−y^2$ into a new function, say, $$g(z) = Re\{f(z)\} = x^2 - y^2 .$$

Then, when you use Tensorflow to evaluate the gradient of $f(z)$ at $z=1j/5$, it turns out that Tensorflow will evaluate the gradient of $g(z)$ but not $f(z)$. So, what tf.gradients returns is $$\nabla_z g(z) = 2 \frac{d g(z)}{dz^*} = 2 \times \underbrace{ \frac{1}{2} \left( \frac{\partial g(z)}{\partial x} + j\frac{\partial g(z)}{\partial y} \right) }_{\textrm{eq.(230) in the book [B1]}} = \frac{\partial g(z)}{\partial x} + j\frac{\partial g(z)}{\partial y} = \underbrace{ \left[ \begin{matrix} \partial g(z)/\partial x \\ \\ \partial g(z)/\partial y \end{matrix} \right] }_{\textrm{in Cartersian coordinates}} .$$

In short, you will end up with $\nabla_z g(z) = 2x + j(-2y)$. As a result, when evaluating $\nabla_z g(z)$ at $z=1j/5$, you will have $\nabla_z g(z)\bigl|_{z=1j/5} \bigr. = 2\times 0 + j(-2\times (1/5)) = -2j/5$. The bottom line is that tf.gradients deal with real-valued functions of complex variables, but not complex-valued functions of complex variables.

[B1] The Matrix Cookbook [ http://matrixcookbook.com ]

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .