I'm interested in evaluating the following quantity $\mathbb{E}\Big[\frac{\partial^2g}{\partial x_1 \partial x_2} \Big]$. g is a function of both variables $x_1,x_2$ which are jointly gaussian in the sense that $(x_1,x_2) \sim \mathcal{N}(\mu,\Sigma)$ with unit variances. That is the diagonal of $\Sigma$ is all 1s and/or $\text{var}(x_1) = \text{var}(x_2) = 1$. g is a non-smooth rectified function as follows: $$g(x_1,x_2) = max(a_1 x_1 + a_2 x_2,0) = \begin{cases} a_1x_1 + a_2 x_2 & \text{if} ~ ~a_1x_1 + a_2 x_2 \ge 0 \\ 0 & \text{else}where\end{cases}$$

By differentiating g w.r.t. $x_1$ we get a Heaviside step function. Differentiating again w.r.t. $x_2$, we end up with a dirac delta function along the line $a_1x_2 + a_2 x_2 = 0$ such that,

$$ \mathbb{E}\Big[ \frac{\partial^2 g}{\partial x_1 \partial x_2}\Big] = \mathbb{E}[\delta(a_1 x_1 + a_2 x_2)] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \delta(a_1 x_1 + a_2 x_2) f(x_1,x_2) dx_1 dx_2 $$

$f(x_1,x_2)$ is the joint Gaussian probability distribution of $(x_1,x_2)$.

I'm not sure if the following is true. Since $\delta(a_1 x_1 + a_2 x_2)$ essentially is a train of impulses along the line $a_1x_2 + a_2 x_2 = 0$, one can write the parameterized dirac function as a two dimensional dirac as $\delta(a_1 x_1 + a_2 x_2) \stackrel{\text{?}}{=} \delta(x_1, -\frac{a_2 x_2}{a_1})$ and by substituting this back in the integral form we get the following:

$$ \mathbb{E}\Big[ \frac{\partial^2 g}{\partial x_1 \partial x_2}\Big] \stackrel{\text{?}}{=} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \delta(x_1, -\frac{a_2 x_2}{a_1}) f(x_1,x_2) dx_1 dx_2 = f(0,0)$$

However, this is counter intuitive. The solution should be the evaluation of the function $f(x_1,x_2)$ along the line $a_1 x_1 + a_2 x_2 = 0$ and not only at a given point.

As for another approach that I have in mind is to parameterize the line $a_1 x1 + a_2 x_2$ in a way that the dirac is a function of only a single variable. This however requires to re-parameterize the PDF $f(x_1,x_2)$ too in a meaningful manner where:

$$f(x_1,x_2) = \frac{1}{2 \pi} \frac{1}{\sqrt{(1- \rho^2)}} exp(-\frac{1}{2 (1 -\rho^2)} \big( (x - \mu_x)^2 -2 \rho (x - \mu_x)(y - \mu_y) + (y - \mu_y)^2\big)\Big)$$

I now come to believe the following is true? $$\mathbb{E}\Big[ \frac{\partial^2 g}{\partial x_1 \partial x_2}\Big] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \delta(a_1 x_1 +a_2 x_2) f(x_1,x_2) dx_1 dx_2 \stackrel{\text{?}}{=} \int_{-\infty}^{\infty} f(-\frac{a_2 x_2}{a_1},x_2) dx_2$$

EDIT: I believe the previous equality is not true as it is missing a scaling factor. By writing the taylor expansion of $a_1 x_1 + a_2 x_2 = 0$ around the it's zeros $a_i$s, one can write $\delta(a_1 x_1 + a_2 x_2) = \sum_i \delta(a_1(x-a_i))$. Also by using the fact that $\int \delta(ax) dx = \int \frac{1}{|a|} \delta(x) dx$ we have the following:

After some treatment I believe the following is true. $$ \mathbb{E}\Big[ \frac{\partial^2 g}{\partial x_1 \partial x_2}\Big] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \delta(a_1 x_1 +a_2 x_2) f(x_1,x_2) dx_1 dx_2 \\ = \int_{-\infty}^{\infty} \sum_i \int_{a_i - \epsilon}^{a_i + \epsilon} \delta(a_1 (x_1 - a_i)) f(x_1,x_2) dx_1 dx_2 \\ = \int_{-\infty}^{\infty} \sum_i \int_{- \epsilon}^{\epsilon} \delta(a_1 y) f(y+a_i,x_2) dy dx_2 \\ = \int_{-\infty}^{\infty} \sum_i \int_{- \epsilon}^{\epsilon} \frac{1}{|a_1|} \delta(y) f(y+a_i,x_2) dy dx_2 \\ = \int_{-\infty}^{\infty} \sum_i \frac{1}{|a_1|} f(a_i,x_2)dx_2 = \int_{-\infty}^{\infty} \frac{1}{|a_1|} f(-\frac{a_2 x_2}{a_1},x_2)dx_2$$

  • $\begingroup$ $\delta(a_1 x_1 + a_2 x_2)$ has its support on a line, while $\delta(x_1, -\frac{a_2 x_2}{a_1})$ has its support in the point $(0, 0)$. $\endgroup$ – md2perpe Jun 14 '17 at 18:44
  • 1
    $\begingroup$ Easier: $\iint \delta(a_1 x_1 + a_2 x_2) f(x_1, x_2) \, dx_1 \, dx_2 \\ = \frac{1}{|a_1|} \iint \delta(x_1 + \frac{a_2 x_2}{a_1}) f(x_1, x_2) \, dx_1 \, dx_2 \\ = \frac{1}{|a_1|} \int f(-\frac{a_2 x_2}{a_1}, x_2) \, dx_2$ $\endgroup$ – md2perpe Jun 15 '17 at 6:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.