Combining multiple variances weighted by probability How do you calculate the overall variance of a situation where there are multiple possible events with known probabilities, and each event has a different and known variance (and expected value)?
As an example, imagine there are three possible carnival games you can play. Each has its own expected value of $E_1$, $E_2$, and $E_3$. Each has its own variance of  $V_1$, $V_2$, $V_3$. You roll a fair die to determine which game you're going to play. If the roll is a 1, you play game 1. If the roll is a 2 or 3, you play game 2. If the roll is a 4, 5, or 6, you play game 3.
What is the variance of this situation?
I would think you just do a weighted average of the three variances. However, that doesn't make sense for the trivial case where all three games have a variance of zero (they always cost/pay a fixed amount) because the weighted average of the variances would be zero, but clearly the overall situation has a non-zero variance with the different expected values for the different games.
 A: You need to compute the weighted average $\mu_1$ and $\mu_2$ of the first two moments $\mu_{1,k}$ and $\mu_{2.k}=V_k+\mu_{1,k}^2$ and then compute the variance $V=\mu_2-\mu_1^2$
$\mu_1=\frac{\sum_{k=1}^3 k\mu_{1,k}}{6}$
$\mu_2=\frac{\sum_{k=1}^3 k\mu_{2,k}}{6}$
Note that the example your questioned. $V=\frac{\sum_{k=1}^3 (k\mu_{1,k})^2}{6}-(\frac{\sum_{k=1}^3 k\mu_{1,k}}{6})^2\ne 0$
A: The situation you stated can be described by the following setup: 
we have three random variables $X_1, X_2,X_3$ representing the three games with respective expected value and variance. Additionally we define a fourth random variable, say $N$, which is independent (I assume rolling the die is independent of the other games)  of the other three and represents the die you roll. We are interested in $Var(X_N)$.
Now let's start with the expected value of $X_N$:
$$
\mathbb{E}[X_N] = \sum_{k=1}^{3} \mathbb{E}[X_N \mathbb{1}\{N=k\}]
=\sum_{k=1}^{3} \mathbb{P}(N=k) \mathbb{E}[X_k] 
$$
where we used the independence of $N$. Similarly we get for the second moment
$$
\mathbb{E}[X_N^2] 
=\sum_{k=1}^{3} \mathbb{P}(N=k) \mathbb{E}[X_k^2]. 
$$
So in both cases the idea of using weighted sums works out, but remember that we use the squared expectation in the formula for the variance.
Therefore, in total the above yields
$$
Var(X_N) = \sum_{k=1}^{3} \mathbb{P}(N=k) \mathbb{E}[X_k^2] - \left( \sum_{k=1}^{3} \mathbb{P}(N=k) \mathbb{E}[X_k]  \right)^2.
$$
Now plug in the probability mass function of $N$ i.e. the die and the respective values for first and second moments of your game.
A: 
I would think you just do a weighted average of the three variances. However, that doesn't make sense for the trivial case where all three games have a variance of zero (they always cost/pay a fixed amount) because the weighted average of the variances would be zero, but clearly the overall situation has a non-zero variance with the different expected values for the different games.

Indeed.   Good work in thinking twice about this.   It is just a little more involved.   There's is an additional component to the total variance.

Let us take $G$ as number for the game, and $R$ as the result for the game.   We are given these measures:
$$\mathsf E(R\mid G)=E_1{\cdot}\mathbf 1_{G=1}+E_2{\cdot}\mathbf 1_{G=2}+E_3{\cdot}\mathbf 1_{G=3}\\\mathsf{Var}(R\mid G)=V_1{\cdot}\mathbf 1_{G=1}+V_2{\cdot}\mathbf 1_{G=2}+V_3{\cdot}\mathbf 1_{G=3}\\\mathsf P(G{=}g)=\tfrac 16{\cdot}\mathbf 1_{g=1}+\tfrac 13{\cdot}\mathbf 1_{g=2}+\tfrac 12{\cdot}\mathbf 1_{g=3}$$

So you should know how to find the Expectation of the Result by the Law of Total Expectation:$$\begin{align}\mathsf E(R)&=\mathsf E(\mathsf E(R\mid G))\\[1ex]&=\sum_g \mathsf P(G{=}g)\cdot\mathsf E(R\mid G{=}g)\\[1ex]&=\tfrac 16 E_1+\tfrac 13 E_2+\tfrac 12E_3\end{align}$$

What you want is to find the Variance of the Result, and to do so you likewise use the Law of Total Variance
$$\mathsf {Var}(R)=\mathsf E(\mathsf{Var}(R\mid G))+\mathsf {Var}(\mathsf E(R\mid G))$$
Now, just apply the definitions for expectation and variance.
$$\begin{align}\mathsf E(\mathsf{Var}(R\mid G))&=\sum_g \mathsf P(G{=}g)\cdot\mathsf {Var}(R\mid G{=}g)\\[1ex]&= \phantom{\tfrac 16V_1+\tfrac 13V_2+\tfrac 12V_3}\\[3ex]\mathsf {Var}(\mathsf E(R\mid G))&=\sum_g \mathsf P(G{=}g)\cdot\mathsf E(R\mid G{=}g)^2-\mathsf E(\mathsf E(R\mid G))^2\\[1ex]&=\phantom{\tfrac 16E_1^2+\tfrac 13E_2^2+\tfrac 12E_3^2-\big(\tfrac 16E_1+\tfrac 13E_2+\tfrac 12E_3\big)^2}\end{align}$$
