Bounds on the expectation of a binary random variable function

Consider R independent binary random variables $Y^1, \ldots, Y^R$ over the space $\{-1,+1\}$ such that $\Pr(Y^j = 1) = p^j \geq 0.5$ and $\Pr(Y^J = -1) = 1 - p^j$, $\forall j = 1, \ldots, R$.

Consider also the following expectation:

\begin{align*} e = E[|\sum_{j=1}^R \text{logit}(p^j) Y^j|] \end{align*} where $\text{logit}(p^j) = \log(\frac{p^j}{1-p^j})$.

By applying the Jensen's inequality it is possible to obtain a lower bound $LB_e$ for $e$:

\begin{align*} E[|\sum_{j=1}^R \text{logit}(p^j) Y^j|] \geq |E[\sum_{j=1}^R \text{logit}(p^j) Y^j]| \end{align*}

How to identify an upper bound $UB_e$ for $e$ as tight as possible? Moreover, is it possible to express $UB_e$ as $k\cdot LB_e$? Thanks.

• With no further information on $(p^j)$, Cauchy-Schwarz inequality might yield the best one can get. – Did Jun 27 '13 at 6:31
• Thank you for your comment. The Cauchy-Scharwz inequality states that $E[XY]^2 \leq E[X^2]E[Y^2]$. In my case, I have a sum of (weighted) random variables, rather than a product. How can I use the inequality in this case? – burton0 Jun 27 '13 at 13:13
• Use $E[|X|]\leqslant E[X^2]^{1/2}$ and expand $X^2=\left(\sum\limits_j\ell_jY^j\right)^2$. – Did Jun 27 '13 at 13:22
• I assume your are using $E[|XY|] \leq \sqrt{E[X^2]E[Y^2]}$ where $Y = 1$ almost surely, right? If this is the case, I don't get how you can go from $|E[XY]| \leq \sqrt{E[X^2]E[Y^2]}$ (Cauchy-Schwarz inequality) to $E[|XY|] \leq \sqrt{E[X^2]E[Y^2]}$. The only way I see is to use the Jensen's inequality, but in that case you have $|E[XY]| \leq E[|XY|]$ and $|E[XY]| \leq \sqrt{E[X^2]E[Y^2]}$ and I don't see how to combine them. – burton0 Jun 27 '13 at 14:33
• Apply C-S to |X| and 1. – Did Jun 27 '13 at 14:41

1 Answer

Hint: if $X$ is a positive random variable, then $E[X]\geq 0$. In our case $X:=|\sum_{j=1}^R l_j Y^{j}|\leq \sum_{j=1}^R |l_j Y^{j}|:=Z$, with $l_j=logit(p_j)$ and so

$$E[Z-X]\geq 0$$

or

$$E[X]\leq E[Z].$$

The expectation value $E[Z]= \sum_{j=1}^R E[|l_j Y^{j}|]$ is an upper bound for $e$.

• Thank you for your answer. The bound you suggest reduces to $\sum_j^R l^j$, which is correct. However, I'm afraid it is not tight enough for my application scenario. Is there a way to tighten the bound? – burton0 Jun 26 '13 at 8:22
• @burtono You are welcome. Feel free to upvote and flag the answer if you accept it. It would be interesting if you could describe your application scenario, in particular the use of logit coefficients. If you want, you can ask for suggestions/opinions using a new question linked to this one. – Avitus Jun 26 '13 at 8:41