Solving a probability inequality Given three probability density functions $p$, $q$ and $h$ of Normal distributions. $p$ and $h$ are fixed and I want to solve for $q$, is it possible to identify a condition on $q$ apart from the trivial cases q=h or q=p such that:
$$\int_{z} \frac{p(z)}{q(z)} \: h(z) \: dz \leqslant 1$$
 A: Here is a partial solution for the special case when $p$ and $h$ have zero mean and different variances:
Suppose $p(z) \sim N(0,\sigma_1^2)$ and $q(z) \sim N(0,\sigma_2^2)$ with $\sigma_1^2\neq \sigma_2^2$.
Define $q_a(z) = a g(az)$ where $g$ is the PDF of a standard normal $N(0,1)$. Then there are distinct positive values $a_1, a_2$ such that $q_{a_1}(z) = p(z)$ and $q_{a_2}(z)=h(z)$. Define
$$\phi(a,z)= \frac{1}{a g(az)}$$
$$ \psi(a) = \int_{-\infty}^{\infty} \phi(a,z)p(z)h(z)dz$$
We have $\psi(a_1)=\psi(a_2)=1$.
For each fixed $z \in \mathbb{R}$ the  function $\phi(a,z)$ is convex in $a$ over the interval $a \in (0,\infty)$:
https://www.wolframalpha.com/input?i=%28d%5E2%2Fda%5E2+1%2F%28a*e%5E%28-a%5E2z%5E2%2F2%29%29+%29%3E%3D0
So the function $\psi(a)$ is convex in $a$.  Thus
$$\psi\left(\frac{a_1+a_2}{2}\right) \leq \frac{1}{2}\psi(a_1) + \frac{1}{2}\psi(a_2)=1$$
So we can use $q_{\frac{a_1+a_2}{2}}(z)$.
A: Another approach: WLOG assume $p \sim N(a, \sigma_1^2)$ and $h \sim N(b, \sigma_2^2)$ where $0<\sigma_1^2 \leq \sigma_2^2$. Define $q\sim N(r, \sigma_2^2)$ for some $r \in \mathbb{R}$ that we will use to achieve the result.
Then
\begin{align}
\frac{h(z)}{q(z)} &= \exp\left(-\frac{(z-b)^2}{2\sigma_2^2}\right)\exp\left(\frac{(z-r)^2}{2\sigma_2^2}\right) \\
&= \exp\left(-\frac{1}{2\sigma_2^2}\left[(z-b)^2 - (z-r)^2 \right]\right)\\
&=\exp\left(\frac{1}{2\sigma_2^2}\left[-2z(b-r) + b^2 -r^2 \right]\right)
\end{align}
So then if we define $\lambda = \sigma_1^2/\sigma_2^2$, we have $0<\lambda \leq 1$ and
\begin{align}
\frac{p(z)h(z)}{q(z)}&=\frac{1}{\sqrt{2\pi\sigma_1^2}}\exp\left(-\frac{(z-a)^2}{2\sigma_1^2}\right)\exp\left(\frac{1}{2\sigma_2^2}\left[-2z(b-r) + b^2 -r^2 \right]\right)\\
&=\frac{1}{\sqrt{2\pi \sigma_1^2}}\exp\left(-\frac{1}{2\sigma_1^2}\left[ (z-a)^2 + 2\lambda z(b-r) -\lambda b^2 + \lambda r^2\right]\right)\\
&=\frac{1}{\sqrt{2\pi \sigma_1^2}}\exp\left(-\frac{1}{2\sigma_1^2}\left[ (z-a+\lambda (b-r))^2 - (-a+\lambda(b-r))^2 - \lambda b^2 + \lambda r^2\right]\right)\\
\end{align}
So
\begin{align}
\int_{-\infty}^{\infty} \frac{p(z)h(z)}{q(z)}dz &= \exp\left(\frac{1}{2\sigma_1^2}\left[(-a+\lambda(b-r))^2 +\lambda b^2- \lambda r^2\right]\right)\\
&=\exp\left(\frac{1}{2\sigma_1^2}\left[ (\lambda^2-\lambda)r^2 + 2ar\lambda + d\right]\right)\\
\end{align}
for some $d \in \mathbb{R}$.  We know $0\leq \lambda \leq 1$. If $0<\lambda<1$ then $\lambda^2-\lambda < 0$ and we can choose $r$ suitably large to make the result strictly less than 1.  If $\lambda=1$ then, if $a\neq 0$, we can again choose $r$ to make the result strictly less than 1.
The only remaining case is if $\lambda=1$ and $a=0$, which gives
\begin{align}
\int_{-\infty}^{\infty} \frac{p(z)h(z)}{q(z)}dz &= \exp\left(\frac{1}{2\sigma_1^2}\left[(-a+\lambda(b-r))^2 +\lambda b^2- \lambda r^2\right]\right)\\
&=\exp\left(\frac{1}{2\sigma_1^2}\left[(b-r)^2+b^2-r^2\right]\right)\\
&=\exp\left(\frac{1}{2\sigma_1^2}\left[2b(b-r)\right]\right)\\
\end{align}
and if $b\neq 0$ we can choose $r$ to make the result strictly less than 1.
The only remaining remaining case is when $\lambda=1$ and $a=b=0$, meaning that we have $p =h\sim N(0,\sigma^2)$.
A: Let $\phi(x; \mu, \sigma^2)$ be the density of $N(\mu, \sigma^2)$. Using the fact
\begin{align*}
\phi(x; \mu_p, \sigma^2_p)\phi(x; \mu_h, \sigma^2_h) = \underbrace{\phi\left(\mu_p; \mu_h,\sigma_p^2 + \sigma_h^2\right)}_{C}\phi(x; \mu, \sigma^2)
\end{align*}
where
\begin{align*}
\mu &= \frac{\sigma_p^{-2}\mu_p + \sigma_h^{-2}\mu_h}{\sigma_p^{-2} + \sigma_h^{-2}} \\
\sigma^2 &= \frac{1}{\sigma_p^{-2} + \sigma_h^{-2}}
\end{align*}
Then we have
\begin{align*}
\int_{\mathbb{R}} \frac{\phi(x; \mu_p, \sigma_p^2)\phi(x; \mu_h, \sigma_h^2)}{\phi(x; \mu_q, \sigma_q^2)}  dx &= C (2\pi \sigma^2_q)^{1/2} \mathbb{E}_{X \sim N(\mu, \sigma^2)}\left[\exp\left(\frac{(X - \mu_q)^2}{2\sigma_q^2}\right)\right] \\
&=C (2\pi \sigma^2_q)^{1/2} \color{blue}{\mathbb{E}_{X \sim N(\frac{\mu-\mu_q}{\sigma}, 1)}\left[\exp\left(X^2\frac{\sigma^2}{2\sigma_q^2}\right)\right]}
\end{align*}
The expression in blue is the MGF of a $\chi^2_1(\frac{\mu-\mu_q}{\sigma})$ evaluated at $\frac{\sigma^2}{2\sigma^2_q}$. Therefore,
\begin{align*}
\int_{\mathbb{R}} \frac{\phi(x; \mu_p, \sigma_p^2)\phi(x; \mu_h, \sigma_h^2)}{\phi(x; \mu_q, \sigma_q^2)}  dx &= C (2\pi \sigma^2_q)^{1/2} \frac{\exp\left(\frac{(\mu-\mu_q)\frac{\sigma}{2\sigma^2_q}}{1-\frac{\sigma^2}{\sigma^2_q}}\right)}{\left(1-\frac{\sigma^2}{\sigma^2_q}\right)^{1/2}} \\
& C \left(2\pi \frac{\sigma^2_q}{1 - \frac{\sigma^2}{\sigma^2_q}}\right)^{1/2} \exp\left(\frac{(\mu-\mu_q)\frac{\sigma}{2}}{\sigma^2_q-\sigma^2}\right)
\\
&\le 1
\end{align*}
You can use this to evaluate combinations of $(\mu_q, \sigma^2_q)$ which satisfy the inequality.
