Gradient of holomorphic function equals conjugate of complex derivative I'm using TensorFlow for some computations with complex variables (and derivatives of these computations). When I compute the derivative of (simple) holomorphic functions, the results obtained with TensorFlow are the conjugate of what I would expect. A simple example:
Given $z = x + yi$, and $f(z) = zz = x^2 + i2xy - y^2$.
We have
$\frac{df}{dz} = \frac12 \left(\frac{\partial f}{\partial x} - i\frac{\partial f}{\partial y}\right) = 2x + i2y$.  
Hence, for $z = \frac{1}{5}i$, $\frac{df}{dz} = \frac{2}{5}i$. However, using TensorFlow, I obtain  $\frac{df}{dz} = -\frac{2}{5}i$. While searching for this, I found the following statement: "The gradient of a holomorphic function is the conjugate of its complex derivative." in a Github Issue, however, I don't understand why. In other words: Why is the gradient of a holomorphic functions equal to the conjugate of the complex derivative, and, where is the mistake in this simple example?
 A: Couple of years late, but I came across this issue too and did some digging.
The key point is that TensorFlow defines the "gradient" of a complex-valued function $f(z)$ of a complex variable as "the gradient of the real map $F: (x,y)\mapsto Re({f(x+iy)})$, expressed as a complex number" (the gradient of that real map is a vector in $\mathbb R^2$, so we can express it as a complex number in the obvious way).
Presumably the reason for that definition is that in TF one is usually concerned with gradients for the purpose of running gradient descent on a loss function, and in particular for identifying the direction of maximum increase/decrease of that loss function. Using the above definition of gradient means that a complex-valued function of complex variables can be used as a loss function in a standard gradient descent algorithm, and the result will be that the real part of the function gets minimised (which seems to me a somewhat reasonable interpretation of "optimise this complex-valued function").
Note that an equivalent way to write that definition of gradient is
$$ gradient(f) := \frac{dF}{dx} + i\frac{dF}{dy}=\overline{\frac{df}{dz} + \frac{d\overline{f}}{dz}},$$
which for a holomorphic function simply reduces to $\overline{\frac{df}{dz}}$.
I wrote up a longer explanation on the GitHub issue you linked: https://github.com/tensorflow/tensorflow/issues/3348#issuecomment-512101921
A: A holomorphic function $z\mapsto f(z)$ has no gradient, but at each point $z$ in its domain $\Omega$ a derivative $f'(z)\in{\mathbb C}$. In this way to $f$  a new function $f': \> z\mapsto f'(z)$ is associated, and it turns out that $f'$ is again holomorphic in $\Omega$.
Now you can write $f(x+iy)=u(x,y)+i v(x,y)$ with real-valued functions $u$ and $v$. In this way $f$ can be viewed as a vector-valued function
$${\bf f}:\quad\Omega\to{\mathbb R}^2, \qquad (x,y)\mapsto\bigl(u(x,y),v(x,y)\bigr)\ .$$
This ${\bf f}$ does not have a gradient either, but at each point $(x,y)\in\Omega$ a Jacobian (or derivative) $d{\bf f}(x,y)$ whose matrix has the special form
$$\left[\matrix{A&-B\cr B&A\cr}\right]\ ,$$
whereby the numbers $A$ and $B$ are related to the derivative of the holomorphic function $f$ we started with by $f'(x+iy)=A+iB$. This is the content of the so-called CR-equations.
A: It is late but I am writing for anyone encountering the same problem as yours. The explanation focuses on why tf.gradients in Tensorflow yeilds (-2j/5) but not (2j/5). First, tf.gradients only handles real-valued functions, thus it converts your original function $f(z)=x^2+2jxy−y^2$ into a new function, say, $$g(z) = Re\{f(z)\} = x^2 - y^2 .$$
Then, when you use Tensorflow to evaluate the gradient of $f(z)$ at $z=1j/5$, it turns out that Tensorflow will evaluate the gradient of $g(z)$ but not $f(z)$. So, what tf.gradients returns is $$\nabla_z g(z) = 2 \frac{d g(z)}{dz^*} = 2 \times \underbrace{ \frac{1}{2} \left( \frac{\partial g(z)}{\partial x} + j\frac{\partial g(z)}{\partial y} \right) }_{\textrm{eq.(230) in the book [B1]}} = \frac{\partial g(z)}{\partial x} + j\frac{\partial g(z)}{\partial y} = \underbrace{ \left[ \begin{matrix} \partial g(z)/\partial x \\ \\ \partial g(z)/\partial y \end{matrix} \right] }_{\textrm{in Cartersian  
coordinates}} .$$
In short, you will end up with $\nabla_z g(z) = 2x + j(-2y)$. As a result, when evaluating  $\nabla_z g(z)$ at $z=1j/5$, you will have $\nabla_z g(z)\bigl|_{z=1j/5} \bigr. = 2\times 0 + j(-2\times (1/5)) = -2j/5$. The bottom line is that tf.gradients deal with real-valued functions of complex variables, but not complex-valued functions of complex variables.
[B1] The Matrix Cookbook [ http://matrixcookbook.com ]
