4
$\begingroup$

Suppose that $f(\theta_1,\dots,\theta_n,x)$ is a (positive) probability density function over a finite set $\mathbb{X}$ defined for an open subset $\Theta\in\mathbb{R}^n$. That is, for every $\theta\in\Theta$ and $x\in\mathbb{X}$, $f(\theta_1,\dots,\theta_n;x)>0$ and $$ \sum_x f(\theta_1,\dots,\theta_n;x) = 1. $$ Let \begin{equation} y_{i,j}(\theta) = \sum_x f(\theta_1,\dots,\theta_n;;x)\frac{\partial}{\partial\theta_i}\log f(\theta_1,\dots,\theta_n;x) \frac{\partial}{\partial\theta_j}\log f(\theta_1,\dots,\theta_n;x). \end{equation} (This is called Fisher information in statistics)

Let $$\widehat{\eta_i}(x) := g_i(\theta_1,\dots,\theta_n;x)$$ and

$$\eta_i := \sum_x f(\theta_1,\dots,\theta_n;x) g_i(\theta_1,\dots,\theta_n;x),$$ for some "nice enough" functions $g_i$. Hence $\eta$ is a function of $\theta$.

It can also be shown that $$ \frac{\partial}{\partial\theta_i}\log f(\theta_1,\dots,\theta_n;x) = \widehat{\eta_i}(x) - \eta_i. $$ By a calculation I am getting $$ \frac{\partial\eta_i}{\partial\theta_j} = y_{i,j}(\theta) + k \cdot \eta_i\eta_j, $$ for some real constant $k$.

However, what I would like to have is $$ \frac{\partial\eta_i}{\partial\theta_j} = y_{i,j}(\theta). $$ So is it possible to modify the $g_i$'s (by scaling/shifting the existing $g_i$) appropriately so that we get the above equation? I tried a bit, couldn't succeed. Any help is greatly appreciated.

$\endgroup$
2
$\begingroup$

Given fixed $i\in \{1,\dots, n\}$ we are looking for a function $\eta_i$ such that

$$\nabla\cdot\eta_i=\left(\frac{\partial\eta_i}{\partial\theta_1},\dots, \frac{\partial\eta_i}{\partial\theta_n}\right)=(y_{i,1},\dots, y_{i,n})=y_i,$$

that is when $y_i$ is a conservative vector field.

I expect this holds when $\Theta$ is simply connected and the functions $y_{i,j}$ are sufficiently smooth (I expect, continuously differentiable). I tried to make this claim more explicit and find a (rigorous) proof for it, but I failed.

Indeed, (assuming $\log=\log_e$) we have $$y_{i,j}=\sum_x \frac 1f\frac{\partial f}{\partial\theta_i}\frac{\partial f}{\partial\theta_j}.$$ Then for each $k\ne j$ we have

$$\frac{\partial y_{i,j}}{\partial\theta_k}=\sum_x -\frac 1{f^2}\frac{\partial f}{\partial\theta_k}\frac{\partial f}{\partial\theta_i}\frac{\partial f}{\partial\theta_j}+ \frac 1f\frac{\partial^2 f}{\partial\theta_i\partial\theta_k}\frac{\partial f}{\partial\theta_j}+ \frac 1f\frac{\partial f}{\partial\theta_j}\frac{\partial^2 f}{\partial\theta_j\partial\theta_k}.$$

If all second order partial derivatives of $f$ with respect to $\theta$’s are continuous then by Schwarz's theorem or Clairaut’s_theorem on equality of mixed partials we have

$\frac{\partial^2 f}{\partial\theta_i\partial\theta_k}=\frac{\partial^2 f}{\partial\theta_k\partial\theta_i}$ and $\frac{\partial^2 f}{\partial\theta_j\partial\theta_k}=\frac{\partial^2 f}{\partial\theta_k\partial\theta_j}$, so $\frac{\partial y_{i,j}}{\partial\theta_k}=\frac{\partial y_{i,k}}{\partial\theta_j}$.

These conditions should be necessarily for $y_i$ to be a conservative vector field (see, for instance, this and this questions).

Moreover, if $\Theta$ is simply connected and the functions $y_{i,j}$ are sufficiently smooth this should be also a sufficient condition. I tried to find an explicit reference for this claim, but I failed beacuse of the following two reasons.

  • Classical multivariable calculus books which I found deal with at most three dimensional case (see the references, in particular, [J, Theorem 6] when $n=2$ and $\Theta$ is a domain, [MI, Theorem 10.2] and [G, Theorem 1.3] when $n=3$), whereas a general case is considered by means of differential forms (see the end of Chapter Irrotational vector fields of the article from Wikipedia and [GW], Sectiton 3 “Differential forms” and Section 5 “The moral of the story”), which, I guess, are proved using Generalizations of Gradient theorem.

  • I am not satisfied with the clarity of formulations and rigor of the proofs from the above references. I found better references from this point of view for continuously differentiable vector fields for $n$ equal to $2$ and $3$ in the books [F, Ch. 15, $\S$ 3] and [LEB, p.340-341], which I studied when I was a student, but they are in Russian and Ukrainian. So I hope other MSE users can provide to you good English references.

On the other hand, procedures to construct potential of a vector field for $n\le 3$ are known, see, for instance, [G, Procedure 1.4], [MI, Section “Finding the Potential, If Possible”], [S, 25.2 Finding Potential Functions], [GW, Subsection “Relating Gradients and Curls” for $n=3$, closed $\Theta$ and continuously differentiable $y_i$].

References

[F] G. Fichtenholz, Differential and Integral Calculus, vol. III, 4-th edition, M.: Nauka, 1968, (in Russian).

[G] R. Garrett, Potential Functions and Conservative Vector Fields.

[GW] E.H. Goins, T.M. Washington, A Tasty Combination: Multivariable Calculus and Differential Forms.

[J] D. Joyce, Conservative vector fields in Math 131 Multivariate Calculus in Clark University, Spring 2014.

[LEB] I. Lyashko, V. Emel’yanov, O. Boyarchuk, Mathematical analysis, vol. 2, Kyiv, Vyshcha shkola, 1993 (in Ukrainian).

[MI] Multidimensional Integration.

[S] T. Silber, Multivariable Calculus Study Guide.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.