# Sum of a stochastic process

I have a question regarding the distribution of the sum of a discrete-time stochastic process. That is, if the stochastic process is $(X_1,X_2,X_3,X_4,\ldots)$, what is the distribution of $X_1+X_2+X_3+\ldots$? $X_i$ could be assumed from a discrete or continuous set, whatever is easier to calculate.

I understand that it mainly depends on the distribution of $X_i$ and on the fact if the $X_i$ are correlated, right? If they are independent, the computation is probably relatively straightforward, right? For the case of two variables, it is the convolution of the probability distributions and probably this can be generalized to the case of n variables, does it? But what if they are dependent?

Are there any types of stochastic processes, where the distribution of the sum can be computed numerically or even be given as a closed-form expression?

I really appreciate any hints!

-
Unclear. Are you thinking of the process $Z_n = \sum_{k=1}^n X_k$ ? Or the sum of the most recent $M$ values ? ($Z_n = \sum_{k={n-M+1}}^n X_k$ ? Furthermore, $Z-n$ is itself a stochastic process (non stationary, at least in the first case), when you speak of its distribution, are you refering to the distribution of $Z_n$ for a single $n$? – leonbloy Aug 16 '12 at 16:21
Thanks for your comment! I am thinking of the process $(X_1, X_2, ...)$ and of the distribution of $Z_n=\sum_{k=1}^n X_k$ for a certain fixed $n$. I don't see the difference in computation of $Z_n=\sum_{k=1}^{n}X_k$ and of $Z_n=\sum_{k=n-M+1}^n X_k$. Is there any? In both cases it is the computation of distribution of the sum of $n$ (possibly depenent) random variables, right? – Konstantin Aug 16 '12 at 17:26
Have you ever considered the discrete-time ito integral (stochastic calculus)? I am on a more general problem that i want to evaluate the r.v. $f(X_1)+ \cdots + f(X_N)$, where $N$ is also an r.v.. And I know that the continuous analogous version is solvable, while i am finding a way to obtain the discrete one... – robit Sep 29 '15 at 12:58

Let $X_1$, $X_2$ be independent (and discrete, for the time being) random variables, and let $Z=X_1+X_2$. Then, to obtain the distribution $P(Z)$ on needs to compute $P(Z=z)$ where $z$ is an arbitrary value:

$$P(Z=z) = \sum_{k=-\infty}^{\infty} P(X_1=k) P(X_2=z-k)$$

This is convolution. It is easy to do the same for the continuous versions of $X_1$ and $X_2$. So, the distribution of the sum of two independent random variables is the convolution of their distributions. This can be generalized to $N$ variables (by adding them one by one).

When $X_1$ and $X_2$ are not independent, then above is not true. To see that, think on how to compute $P(Z=z)$. You would need to use the conditional probabilities, i.e. $P(X_1|X_2)$ or $P(X_2|X_1)$.

When $X_i$s are independent normally distributed random variables, $Z$ is also normally disrtibuted with mean $\sum_i \mu_{X_i}$ and variance $\sum_i \sigma^2_{X_i}$.

There are other distributions where the distribution of the sum can be expressed in closed form. For example, the Cauchy distribution.

-
Thanks for you answer! Actually, the case of two independent random variables was already more or less clear to me, as I wrote in the question. The case I am interested in is the sum of $n$ dependent random variables. Are there some prominent distributions/random processes, for which such a computation can be performed? – Konstantin Aug 16 '12 at 17:31
I am not aware of any generic formula on the sum of dependent random variables. I guess you should further elaborate on what "dependency" means. Write it out: $P(Z=z) = \sum_k P(X=k) P(Y=z-k|X=k)$, can you compute this? In special cases, for example, the distribution of sum of correlated normal random variables should be easy to compute. – emrea Aug 16 '12 at 17:48

Are there any types of stochastic processes, where the distribution of the sum can be computed numerically or even be given as a closed-form expression?

As stated, the problem is quite equivalent to compute the distribution of the sum of an arbritary set of random variables. Little can be said in general, as the fact that the variables ($X_i$) form a stochastic process adds practically nothing.

Let's assume that the stochastic process $X(n)$ is a stationary ARMA$(P,Q)$ process, i.e., it's generated from a white noise process $R(n)$ of zero mean and given distribution that passes through a LTI causal filter with $P$ zeroes and $Q$ poles. Then, the process $Z(n) = \sum_{k=n-M+1}^{n} X(k)$ is obtained by chaining a MA$(M)$ filter, so $Z(n)$ is ARMA$(P+M,Q)$ (apart from cancellation which might occur). Now any finite order invertible causal ARMA filter can be expressed as an infinite order MA filter, so that $Z(n)$ can be expressed as a (infinite) linear combination of the white noise input:

$$Z(n) = \sum_{k=-\infty}^n a_k R(k)$$

Because $R(k)$ is iid, the distribution of the sum can be obtained as a convolution. (Notice, however, that the CLT does not apply here). In terms of the characteristic functions, we'd get

$$F_Z(w)=\prod_{k=-\infty}^n F_R(a_k w)$$

Notice, however, that all this might have little or not practical use. For one thing, ARMA modelling is usually applied only to second order moment analysis.

-

I believe a general method is to find characteristic functions of $X_i$, then the characteristic function of the sum is easy to compute. The difficult part is finding the inverse Fourier transform. If you're interested in numerical approximation, inverse Fourier transform can be computed numerically. Also, if you accept approximation, the central limit theorem is not a bad thing to use.

-
"then the characteristic function of the sum is easy to compute" Also when the $X_i$ are correlated? – leonbloy Aug 16 '12 at 19:50