sum of independent exponential random variables

Let $X$ be the sum of two independent exponential random variables: $X_{1}$ with parameter $\lambda_{1} = \frac{1}{5}$ and $X_{2}$ with parameter $\lambda_{2} = 2$. These random variables have values in the interval $[0,60]$

I want to prove the variance of $X$ is $401$. I found by looking into some references that the p.d.f of $X$ is: $$f_{X}(t) = \sum_{i=1}^{2} {\frac{\lambda_{1} \lambda_{2} }{\prod_{j=1,j \ne i}^{2}(\lambda_{j} -\lambda_{i})} exp(-\lambda_{i} t)}$$

I wanted to know if the above density is correct or not since I did find a value different than what I'm supposed to get for the variance computation.

Thanks.

-
What is the mean of $X_1?$ What is its variance? You have not made the parameterization clear. Regardless, I don't see how 401 can be correct. – soakley Apr 26 '14 at 13:46
I did say that $X_{1}$ is an exponential random variable. So, the mean and the variance are respectively $\frac{1}{\lambda_{1}}$ and $\frac{1}{\lambda_{1}^{2}}$ – kagami Apr 26 '14 at 15:06
@kagami Perhaps you are not aware of this fact, but the exponential distribution is frequently characterized in two different ways: one way is such that the parameter is the reciprocal of the expected value (as you have done here; this is called the rate parametrization), and another is where the parameter is the expected value (scale parametrization): ${\rm E}[X] = \lambda$, so its PDF is $$f_X(x) = \frac{1}{\lambda}e^{-x/\lambda}, \quad x > 0.$$ Without specifying which one is being used, it is inappropriate to assume that others know which one you mean. – heropup Apr 26 '14 at 15:21
Thanks for you comment. I wasn't aware of this fact but there was no explicit mention of what type of characterization, I know that the variables are exponential and since I knew only the characterization mentioned above, I assume that this is the right characterization. – kagami Apr 26 '14 at 15:55

I'm adding this as a separate answer as it is somewhat unrelated to my original post, and is quite lengthy in itself.

My previous post addresses the question originally asked, though it became clear in the comments that the question of interest is actually the following (taken from the comments):

The independent visitors of a certain Web site may be divided into two groups: those who arrived on this site voluntarily (type 1) and those who arrived there by chance or by error (type 2). Let N(t) be the total number of visitors in the interval [0,t]. We suppose that {N{t),t > 0} is a Poisson process with rate a=10 per hour, and that 80 of the visitors are of type 1 (and 20 of type 2).The question was: "Calculate the variance of the total time spent on this site by the visitors arrived in the interval [0,1] if the time (in minutes) X1 (respectively, X2) that a type 1 (resp., type 2) visitor spends on the site in question is an exponential random variable with parameter 15 (resp., 2). Moreover, we assume that both varibales are independent random variables.

I will highlight two approaches to the problem: one working with knowledge of independent variables only and Wald's equation, and the second using properties of the Poisson and Exponential distributions. Before we get to these we make the following note:

If the arrivals of a rate $\lambda$ Poisson process can be split into two types (types 1 and 2) with the proportion of type 1 arrivals being $p_1$, and type 2 occurring with $p_2 = 1 -p_1$, then the process can be `thinned' into two sub-processes with rates $\lambda_1 = \lambda p_1$, and $\lambda_2 = \lambda p_2$ respectively. These two processes are independent of each other by properties of Poisson processes. In your question, you have $\lambda = 10$ (which I interpret as an average of 10 visitors to the site per hour), and $\lambda_1 = 0.8 \times 10 = 8$, and $\lambda_2 = 0.2 \times 10 = 2$.

The question asks you to compute the variance of the total time spent by all visitors in 1 hour, where the arrivals are modeled by a Poisson process. Let $T_i$ denote the time spent by the $i$-th arrival, and $N$ the total number of arrivals in an hour, so we consider \begin{align*} T = \sum_{i=1}^N T_i. \end{align*} There are currently two problems to face: the first is that the $T_i$ are all exponential (as stated in the question), but with different parameters. The second is the fact that $N$ is itself a random variable. The solution to the first of these problems is to use our thinning relation described above: viewing the process with rate $\lambda$ as the union of the $\lambda_1$ and $\lambda_2$ processes then: $N = N_1 + N_2$, and then \begin{align*} T = \sum_{i=1}^{N_1} T_i^{(1)} + \sum_{i=1}^{N_2} T_i^{(2)}, \end{align*} where as given in the question $T_i^{(1)}$ are exponential of parameter $\tau_1$, and $T_i^{(2)}$ are exponential of parameter $\tau_2$ (as given: $\tau_1 = 2$, $\tau_2 = 1/5$). According to the formula for the variance which was given in my previous post \begin{align*} \text{Var}[T] = \text{Var}\Big[ \sum_{i=1}^{N_1} T_i^{(1)} \Big]+ \text{Var}\Big[ \sum_{i=1}^{N_2} T_i^{(2)} \Big]. \end{align*} So our problem is now about taking variances of a sum of randomly many random variables.

In the following I drop the sub-script $N_1,N_2$ since the above formula tells you how to apply this for your problem. So $N$ is now a Poisson variable of rate $\lambda$, and each $T_i$ is exponential with rate $\tau$.

The first approach is to find the variance of such a sum is to use a result called Wald's Equation, which says that under the assumption of independence of $N$ from the $T_i$, and if the $T_i$ are i.i.d. then \begin{align*} \mathbf E [ \sum_{i=1}^N T_i] = \mathbf E[N] \mathbf E[T_1] \end{align*} One can quickly extend this to a theorem about the variance instead: \begin{align*} \text{Var} [ \sum_{i=1}^N T_i] = \text{Var}[N]\mathbf E[T_1]^2 + \mathbf E[N] \text{Var}[T_1]\end{align*} You can now apply this to the case we have with $N$ a Poisson variable, independent of the $T_i$ which are i.i.d. exponential variables.

Of course, this method relies on you knowing Wald's equation, and you would probably want to be able to derive this too. So, how can we solve the problem without using that? The alternative is to use conditional expectations, and the fact that we know the distributions of the relevant variables. I'll do the calculation for the expectation of $\sum_{i=1}^N T_i$, and leave you to calculate the second moment (which will be needed to find the variance).

So if $N \sim \text{Poisson}(\lambda)$, and $T_i \sim \text{Exp}(\tau)$, then conditioning on the value $N$ takes \begin{align*} \mathbf E \Big[ \sum_{i=1}^N T_i ] & = \sum_{n=0}^N \mathbf E \Big[ \sum_{i=1}^N T_i | N = n ] \mathbf P[ N = n] \\ & = \sum_{n=0}^\infty \mathbf E \Big[ \sum_{i=1}^n T_i ] \mathbf P[ N = n] \\ & = \sum_{n = 0}^\infty n \mathbf E[\tau] \mathbf P[ N = n] \\ & = \mathbf E[\tau] e^{-\lambda} \sum_{n = 0}^\infty n \lambda^n \frac{1}{n!} \\ & = \frac{\lambda}{\tau}. \end{align*} Note that this agrees with the answer we would get from Wald's equation.

That should now be enough to let you answer the question.

-
thanks for your detailed answer. I only wanted to check with you since I found out that the extension of Wald's equation for the variance computation is written as: \begin{align*} \text{Var} [ \sum_{i=1}^N T_i] = \text{Var}[N] \mathbf E[T_1]^2 + \mathbf E[N] \text{Var}[T_1]\end{align*} – kagami Apr 27 '14 at 15:24
Yes that's correct: that was a typo on my behalf. Corrected. – owen88 Apr 27 '14 at 15:36

There are a few points to be addressed in your question. First of all, exponential distributions are supported on the entirety of the positive real line, meaning that $X_1, X_2$ take values in $[0,\infty)$, rather than $[0,60]$ as you claim; moreover their sum $X = X_1 + X_2$ also takes values in $[0,\infty)$.

There are two immediate approaches to calculate the variance of $X$. The first one depends only on the fact that they are independent. A basic fact in probability theory asserts that if $U, V$ are independent random variables, then

\begin{align*} \text{Var}(U+V) &= \mathbf E[ (U+ V)^2] - \mathbf E[U + V]^2 \\ & = \mathbf E[U^2] + \mathbf E[V^2] + 2\mathbf E[U]\mathbf E[V] - (\mathbf E[U]^2 + \mathbf E[V]^2 + 2\mathbf E[U]\mathbf E[V] )\\ & = \text{Var}[U] + \text{Var}[V] \end{align*}

From this it follows from the fact that the variance of an $\text{Exp}(\lambda)$ variable is $\lambda^{-2}$, that \begin{align*} \text{Var}(X_1 + X_2) & = \lambda_1^{-2} + \lambda_2^{-2} \\ & = \frac{101}{4}. \end{align*} for $\lambda_1 = 1/5$, $\lambda_2 = 2$.

Note that in this approach we did not need any properties of the distributions, other than knowledge of their variances (i.e. if you gave me two distributions $U,V$, with $\text{Var}[U] = \lambda_1, \, \text{Var}[V] = \lambda_2$, the answer would not change).

A second approach would be to argue via the probability density functions (p.d.f.s), as you suggest in the question. This will in general be a harder approach, as we will have to derive the variance of the sum from the p.d.f., as well as computing the p.d.f.

In general for independent $U, V$, with p.d.f. $f_U, f_V$ respectively, the distribution of their sum is given as the convolution of their p.d.f.s \begin{align*} f_{U +V}(x) = \int_{-\infty}^\infty f_U(y)f_V(x-y)dy. \end{align*} In the case of a exponential distributions, the p.d.f. of a $\lambda$ exponential distribution is \begin{align*} f(x) = \lambda e^{-x} \end{align*} And so for two independent exponentials with parameters $\lambda_1 \neq \lambda_2$ \begin{align*} f_{X}(x) &= f_{X_1 + X_2}(x) \\ & = \int_{0}^x \lambda_1 e^{-\lambda_1 y} \lambda_2 e^{-\lambda_2(x-y)}dy \\ & = \lambda_1 \lambda_2 e^{-\lambda_2 x}\int_{0}^x e^{(\lambda_2-\lambda_1)y}dy \\ & = \frac{\lambda_1 \lambda_2}{\lambda_2-\lambda_1}( e^{-\lambda_1 x}-e^{-\lambda_2 x}). \end{align*} Note that we only integrate over the region supported by the p.d.f.s. From this we can then proceed to calculate the variance from the p.d.f. by the standard formulae \begin{align*} \mathbf E[V] &= \int_{-\infty}^\infty x f_V(x) d x,\\ \mathbf E[V^2] &= \int_{-\infty}^\infty x^2 f_V(x) dx \end{align*}

In all, I hope that my post has made it clear that this second approach requires a lot more work, and is more prone to calculation error!

-
I'm solving an exercise from a book since I'm not familiar with probability and I'm practicing. – kagami Apr 26 '14 at 15:48
The independent visitors of a certain Web site may be divided into two groups: those who arrived on this site voluntarily (type $1$) and those who arrived there by chance or by error (type $2$). Let $N(t)$ be the total number of visitors in the interval [0,t]. We suppose that {N{t),t > 0} is a Poisson process with rate $a = 10$ per hour, and that $80%$ of the visitors are of type $1$ (and $20%$ of type $2$). – kagami Apr 26 '14 at 15:49
The question was: "Calculate the variance of the total time spent on this site by the visitors arrived in the interval $[0,1]$ if the time (in minutes) $X_{1}$ (respectively, $X_{2}$) that a type $1$ (resp., type $2$) visitor spends on the site in question is an exponential random variable with parameter $\frac{1}{5}$ (resp., $2$). Moreover, we assume that both varibales are independent random variables. – kagami Apr 26 '14 at 15:51
Okay, it seems to me that your original question is not the correct question to ask. Can you please give me a reference to the book you're following as your question still seems somewhat ambiguous. The quick answer I can give is it seems as though you need to use Poisson thinning to answer the question, but from the way it is phrased above, I'm not certain I understand the question at all. In particular: are we to calculate the variance of the time spent by all arrivals, or the variance of a single arrival during that time? – owen88 Apr 26 '14 at 16:02
Okay: so the answer to this question is somewhat more involved than the question you originally stated was. My answer above answers the question you stated. To the best of my knowledge, the question in the book will first require you to understand the idea of Poisson thinning, and also you'll need to understand conditional expectations (these are necessary as you do not know how manyarrivals you're going to need to sum over). I will edit my answer above, but it will take a while...! – owen88 Apr 26 '14 at 16:14