# Multivariable Chain Rule: Conditional Independence and "Swapping" Partial Derivatives

Suppose that we have three functions $$f$$, $$g$$, and $$h$$, such that for all $$x$$ and $$y$$, $$h(x, y) = g(f(x, y, \cdot))$$, where $$\cdot$$ represents an input where a random variable goes (ignored in the informal version below), and $$g$$ is the expectation of an expression involving that random variables (this makes $$g$$ and $$h$$ deterministic functions). We are given $$\frac{\partial h(x, y)}{\partial x}$$, and want to find $$\frac{\partial h(x, y)}{\partial y}$$.

Informal Version (more formal version given below):

We are given the partial derivative of $$h$$ w.r.t. $$x$$,

$$\frac{\partial h(x, y)}{\partial x} = \mathbb E \left[\sum_{t=0}^\infty \text{[some stuff involving random variables and } f(x, y)] \frac{\partial f(x, y)}{\partial x}\right],$$

Where the "some stuff" is all independent of $$x$$ and $$y$$ given the output of $$f(x, y)$$. We want to prove that we can substitute $$\partial y$$ for $$\partial x$$:

$$\frac{\partial h(x, y)}{\partial y} = \mathbb E \left[\sum_{t=0}^\infty \text{[some stuff involving random variables and } f(x, y)] \frac{\partial f(x, y)}{\partial y}\right].$$

Intuition/Main Question:

Intuitively, we can argue that since $$x$$ and $$y$$ only affect $$h$$ via $$f$$ (that is, $$h$$ is conditionally independent of $$x$$ and $$y$$ given $$f(x, y)$$), and we should be able to simply replace $$\frac{\partial f(x, y)}{\partial x}$$ with $$\frac{\partial f(x, y)}{\partial y}$$ in the definition of $$\frac{\partial h(x, y)}{\partial x}$$ to get $$\frac{\partial h(x, y)}{\partial y}$$ (see the informal chain rule argument in the appendix below for more intuition).

How can we formalize the argument above to prove this property about $$\frac{\partial h(x, y)}{\partial y}$$? I suspect there some theorem or well-known result that immediately gives us what we need, but I do not know what this theorem is. Thank you.

# Appendix (not necessary for answering the question, but may be helpful):

More Formal Version:

We are given that partial derivative of $$h$$ w.r.t. $$x$$,

$$\frac{\partial h(x, y)}{\partial x} = \mathbb E \left[\sum_{t=0}^\infty d(f(x, y, Z_t), Z_t) \frac{\partial f(x, y, Z_t)}{\partial x}\right],$$

where, $$d$$ is some function, the $$Z_t$$'s are random variables influenced by $$f(x, y, \cdot)$$ and, for all $$t$$, $$Z_t$$ is conditionally independent of $$x$$ and $$y$$ given $$Z_{t-1}$$ and $$f(x, y, Z_{t-1})$$. We need to find $$\frac{dh(x, y)}{dy}.$$ We'd like to show that

$$\frac{\partial h(x, y)}{\partial y} = \mathbb E \left[\sum_{t=0}^\infty d(f(x, y, Z_t), Z_t) \frac{\partial f(x, y, Z_t)}{\partial y}\right].$$

$$\frac{\partial h}{\partial x} = \frac{\partial h}{\partial f} \frac{\partial f}{\partial x} = d(f(x, y)) \frac{\partial f(x, y)}{\partial x},$$
so $$\frac{\partial h}{\partial f} = d(f(x, y)).$$ Substituting this definition of $$\frac{\partial h}{\partial f}$$ to find $$\frac{\partial h}{\partial y}$$:
$$\frac{\partial h}{\partial y} = \frac{\partial h}{\partial f} \frac{\partial f}{\partial y} = d(f(x, y)) \frac{\partial f(x, y)}{\partial y}.$$