# Derivative of a multivariable function (arises in mathematical statistics)

Suppose that $$f(\theta_1,\dots,\theta_n,x)$$ is a (positive) probability density function over a finite set $$\mathbb{X}$$ defined for an open subset $$\Theta\in\mathbb{R}^n$$. That is, for every $$\theta\in\Theta$$ and $$x\in\mathbb{X}$$, $$f(\theta_1,\dots,\theta_n;x)>0$$ and $$\sum_x f(\theta_1,\dots,\theta_n;x) = 1.$$ Let $$\begin{equation} y_{i,j}(\theta) = \sum_x f(\theta_1,\dots,\theta_n;;x)\frac{\partial}{\partial\theta_i}\log f(\theta_1,\dots,\theta_n;x) \frac{\partial}{\partial\theta_j}\log f(\theta_1,\dots,\theta_n;x). \end{equation}$$ (This is called Fisher information in statistics)

Let $$\widehat{\eta_i}(x) := g_i(\theta_1,\dots,\theta_n;x)$$ and

$$\eta_i := \sum_x f(\theta_1,\dots,\theta_n;x) g_i(\theta_1,\dots,\theta_n;x),$$ for some "nice enough" functions $$g_i$$. Hence $$\eta$$ is a function of $$\theta$$.

It can also be shown that $$\frac{\partial}{\partial\theta_i}\log f(\theta_1,\dots,\theta_n;x) = \widehat{\eta_i}(x) - \eta_i.$$ By a calculation I am getting $$\frac{\partial\eta_i}{\partial\theta_j} = y_{i,j}(\theta) + k \cdot \eta_i\eta_j,$$ for some real constant $$k$$.

However, what I would like to have is $$\frac{\partial\eta_i}{\partial\theta_j} = y_{i,j}(\theta).$$ So is it possible to modify the $$g_i$$'s (by scaling/shifting the existing $$g_i$$) appropriately so that we get the above equation? I tried a bit, couldn't succeed. Any help is greatly appreciated.

Given fixed $$i\in \{1,\dots, n\}$$ we are looking for a function $$\eta_i$$ such that

$$\nabla\cdot\eta_i=\left(\frac{\partial\eta_i}{\partial\theta_1},\dots, \frac{\partial\eta_i}{\partial\theta_n}\right)=(y_{i,1},\dots, y_{i,n})=y_i,$$

that is when $$y_i$$ is a conservative vector field.

I expect this holds when $$\Theta$$ is simply connected and the functions $$y_{i,j}$$ are sufficiently smooth (I expect, continuously differentiable). I tried to make this claim more explicit and find a (rigorous) proof for it, but I failed.

Indeed, (assuming $$\log=\log_e$$) we have $$y_{i,j}=\sum_x \frac 1f\frac{\partial f}{\partial\theta_i}\frac{\partial f}{\partial\theta_j}.$$ Then for each $$k\ne j$$ we have

$$\frac{\partial y_{i,j}}{\partial\theta_k}=\sum_x -\frac 1{f^2}\frac{\partial f}{\partial\theta_k}\frac{\partial f}{\partial\theta_i}\frac{\partial f}{\partial\theta_j}+ \frac 1f\frac{\partial^2 f}{\partial\theta_i\partial\theta_k}\frac{\partial f}{\partial\theta_j}+ \frac 1f\frac{\partial f}{\partial\theta_j}\frac{\partial^2 f}{\partial\theta_j\partial\theta_k}.$$

If all second order partial derivatives of $$f$$ with respect to $$\theta$$’s are continuous then by Schwarz's theorem or Clairaut’s_theorem on equality of mixed partials we have

$$\frac{\partial^2 f}{\partial\theta_i\partial\theta_k}=\frac{\partial^2 f}{\partial\theta_k\partial\theta_i}$$ and $$\frac{\partial^2 f}{\partial\theta_j\partial\theta_k}=\frac{\partial^2 f}{\partial\theta_k\partial\theta_j}$$, so $$\frac{\partial y_{i,j}}{\partial\theta_k}=\frac{\partial y_{i,k}}{\partial\theta_j}$$.

These conditions should be necessarily for $$y_i$$ to be a conservative vector field (see, for instance, this and this questions).

Moreover, if $$\Theta$$ is simply connected and the functions $$y_{i,j}$$ are sufficiently smooth this should be also a sufficient condition. I tried to find an explicit reference for this claim, but I failed beacuse of the following two reasons.

• Classical multivariable calculus books which I found deal with at most three dimensional case (see the references, in particular, [J, Theorem 6] when $$n=2$$ and $$\Theta$$ is a domain, [MI, Theorem 10.2] and [G, Theorem 1.3] when $$n=3$$), whereas a general case is considered by means of differential forms (see the end of Chapter Irrotational vector fields of the article from Wikipedia and [GW], Sectiton 3 “Differential forms” and Section 5 “The moral of the story”), which, I guess, are proved using Generalizations of Gradient theorem.

• I am not satisfied with the clarity of formulations and rigor of the proofs from the above references. I found better references from this point of view for continuously differentiable vector fields for $$n$$ equal to $$2$$ and $$3$$ in the books [F, Ch. 15, $$\S$$ 3] and [LEB, p.340-341], which I studied when I was a student, but they are in Russian and Ukrainian. So I hope other MSE users can provide to you good English references.

On the other hand, procedures to construct potential of a vector field for $$n\le 3$$ are known, see, for instance, [G, Procedure 1.4], [MI, Section “Finding the Potential, If Possible”], [S, 25.2 Finding Potential Functions], [GW, Subsection “Relating Gradients and Curls” for $$n=3$$, closed $$\Theta$$ and continuously differentiable $$y_i$$].

References

[F] G. Fichtenholz, Differential and Integral Calculus, vol. III, 4-th edition, M.: Nauka, 1968, (in Russian).

[G] R. Garrett, Potential Functions and Conservative Vector Fields.

[GW] E.H. Goins, T.M. Washington, A Tasty Combination: Multivariable Calculus and Differential Forms.

[J] D. Joyce, Conservative vector fields in Math 131 Multivariate Calculus in Clark University, Spring 2014.

[LEB] I. Lyashko, V. Emel’yanov, O. Boyarchuk, Mathematical analysis, vol. 2, Kyiv, Vyshcha shkola, 1993 (in Ukrainian).

[S] T. Silber, Multivariable Calculus Study Guide.