# Application of the Jensen's inequality

For a convex function $$\phi(x)$$, Jensen's inequality states: $$$$E_{x}(\phi(x)) \geq \phi(E_{x}(X))$$$$ where $$E_x$$ is the expectation. When applying the Jensen's inequality to the following expression $$$$\left(E_{z} \left \{E_{y|z} \left\{\int_{\mathbb{R}} f^{r}(x|y,z)\,dx\right\}\right\}\right)^{\frac{1}{1-r}}$$$$ assuming $$0 the function $$\frac{1}{1-r} >1$$ so $$$$\left(E_{z} \left \{E_{y|z} \left\{\int_{\mathbb{R}} f^{r}(x|y,z)\,dx\right\}\right\}\right)^{\frac{1}{1-r}} \leq E_{z} \left\{ \left(E_{y|z} \left\{\int_{\mathbb{R}} f^{r}(x|y,z)\,dx\right\}\right)^{\frac{1}{1-r}}\right\}$$$$ I think the result should be just the opposite (i.e. $$\leq$$ should be replaced by $$\geq$$), but I don't understand why. Could you please help me? When looking at the expression above we realize that $$$$E_{z} \left\{ \left(E_{y|z} \left\{\int_{\mathbb{R}} f^{r}(x|y,z)\,dx\right\}\right)^{\frac{1}{1-r}}\right\} =\mathcal{E}_{r}(X|Y,Z)$$$$ So if the relation I wrote is correct $$$$\mathcal{E}_{r}(X|Y,Z) \geq \left(E_{z} \left \{E_{y|z} \left\{\int_{\mathbb{R}} f^{r}(x|y,z)\,dx\right\}\right\}\right)^{\frac{1}{1-r}} = \left(E_{z} \left\{\int_{\mathbb{R}} f^{r}(x|z)\,dx\right\}\right)^{\frac{1}{1-r}}$$$$ By applying the Jensen's inequality to the rhs we have $$$$\mathcal{E}_{r}(X|Z) = E_{z} \left \{\left(\int_{\mathbb{R}} f^{r}(x|z)\,dx\right)^{\frac{1}{1-r}} \right \} \geq \left(E_{z} \left \{ \int_{\mathbb{R}} f^{r}(x|z)\,dx\right\}\right)^{\frac{1}{1-r}}$$$$ I expect to get $$$$\mathcal{E}_{r}(X|Y,Z) \leq \mathcal{E}_{r}(X|Z)$$$$ which is possible only if the sign is reversed.As a matter of fact conditioning on more variables (Y and Z) can only reduce the entropy compared to conditioning on fewer variables (only Z).

• Why do you think it should be ?
– EDX
Commented Jun 9 at 18:28
I think you are looking at the wrong function. $$$$\mathcal{E}_r(X|Z) = E_Z \left\{ \left( \int_\mathbb{R} f^r(x|z) \, dx \right)^{\frac{1}{1-r}} \right\} = E_Z \left\{ \left( \int_\mathbb{R} \left( E_{Y|Z} \left[ f(x|y,z) \right] \right)^r \, dx \right)^{\frac{1}{1-r}} \right\}.$$$$ Assuming $$0 the function $$\phi(x) = x^r$$ is concave, so $$$$\left( E_{Y|Z} \left[ f(x|y,z) \right] \right)^r \ge E_{Y|Z} \left[ f^r(x|y,z) \right]$$$$ $$$$\mathcal{E}_r(X|Z) \ge E_Z \left\{ \left( \int_\mathbb{R} E_{Y|Z} \left[ f^r(x|y,z) \right] \, dx \right)^{\frac{1}{1-r}} \right\} = \mathcal{E}_r(X|Y,Z).$$$$