Let $\vec{b}$ denote a vector of $n$ Bernoulli variables and let $A \in \mathbb R ^{n\times n}$ be a constant matrix. Can we compute the following expectation:

$$\mathbb E[ \mathrm{diag}(\vec{b}) A \,\mathrm{diag}(\vec{b})],$$

taken over the joint distribution of $\vec{b}$, which is:

$$p(\vec{b}) = \prod_i^n a_i^{b_i} (1-a_i)^{1-b_i},$$

where $a_i$ are the parameters of the individual Bernoulli distributions? Note that the expectation is also a matrix $n \times n$ and it should be expressed in terms of the parameters $a_i$.

  • $\begingroup$ Just to verify my understanding: $\mathrm{diag}(\vec{b})$ is a diagonal $n \times n$ matrix, right? (Its diagonal entries $\in \{0,1\}$.) So $\mathrm{diag}(\vec{b}) A \,\mathrm{diag}(\vec{b})$ is also an $n \times n$ matrix, and you want its expectation, which is also an $n \times n$ matrix, right? $\endgroup$ – antkam Apr 18 '18 at 14:45
  • $\begingroup$ Correct. I'm looking for the expectation in terms of the parameters $a$ (will update the question) $\endgroup$ – Jan Kukacka Apr 18 '18 at 16:28

Let $B = \mathrm{diag}(\vec{b})$, which is a diagonal matrix with diagonal entries $\in \{0, 1\}$.

For any matrix $M$, denote by $M_{*j}$ the $j$th column of $M$ (which is an $n \times 1$ column vector). E.g., $B_{*j}$ is either $0$ or $1$ at its $j$th position, but $0$ everywhere else. We have:

  • if $b_j = 0$ then $(AB)_{*j} = A \times B_{*j} = 0$

  • if $b_j = 1$ then $(AB)_{*j} = A \times B_{*j} = A_{*j}= j$th column of $A$

I.e. $AB$ consists of those $j$th columns of $A$ where $b_j=1$. The columns where $b_j = 0$ have been replaced with $0$.

Similarly, Left-multiplying by $B$ picks out those $i$th rows where $b_i = 1$.

Thus, $(BAB)_{ij} = b_i A_{ij} b_j$, i.e. it is the original $A_{ij}$ entry iff both $b_i=b_j=1$, and $0$ otherwise.

If the above is not convincing enough :) go to algebra:

$$(BAB)_{ij} = \sum_k \sum_l B_{ik} A_{kl} B_{lj} = B_{ii} A_{ij} B_{jj}$$

where the last equality is due to $B$ being diagonal and so all $B_{ik}$ vanish except for $k=i$, and all $B_{lj}$ vanish except for $B_{jj}$.

Finally, $E[(BAB)_{ij}] = E[b_i A_{ij} b_j]$. Now using $P(b_i = 1) = a_i$, we have:

  • For the diagonal entries, $b_i = b_j$ is the same random variable, so $E[(BAB)_{ii}] = a_i A_{ii}$

  • For the non-diagonal entries, $b_i, b_j$ are two independent random variables, so $E[(BAB)_{ij}] = a_i A_{ij} a_j$.

Incidentally, the non-diagonal entries are the same as those in $C=\mathrm{diag}(\vec{a}) A\,\mathrm{diag}(\vec{a})$. The diagonal entries are different though, because $C_{ii} = a_i A_{ii} a_i = a^2_i A_{ii} \neq a_i A_{ii}$.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.