what does ensemble average mean?

I'm studying this paper and somewhere in the conclusion part is written:

"Since this rotation of the coherency matrix is carried out based on the ensemble average of polarimetric scattering characteristics in a selected imaging window, we obtain the rotation angle as a result of second-order statistics."

Also I've seen the term ensemble average in several other papers of this context.

Now I want to understand the exact mathematical or statistical definition of ensemble averaging not only in this context but the exact meaning and use of ensemble averaging in statistics and mathematics.

I googled the term ensemble average and here in wikipedia we have the definition as

"In statistical mechanics, the ensemble average is defined as the mean of a quantity that is a function of the microstate of a system (the ensemble of possible states), according to the distribution of the system on its microstates in this ensemble."

But I didn't understand this definition because I don't even know what does the microstate of a system or possible states of system mean in mathematics.

Could you please give me a simple definition with some examples for ensemble averaging?
Compare time averaging and ensemble averaging?
And also introduce me some good resources to study more especially resources that can be helpful in image processing too?

• Are you asking for a rephrasing with less physics jargon? What system of jargon would you prefer? I can give a description in dynamical systems jargon if you want.
– Ian
Jun 25 '15 at 16:04
• I found a course on the net ocw.mit.edu/courses/electrical-engineering-and-computer-science/… I'll download the 13th lecture, watch it and then improve my question @lan Jun 25 '15 at 16:19
• @lan it seems that you need more information to answer my question. I'm working on SAR images so I prefer a description in image processing jargon but first I want to know the discription purely in mathematic and statistics. I'll be back soon with improvements to my question Jun 25 '15 at 16:22

I realize this is a late answer to this post, but it still makes the top two to three results on Google for "ensemble average" and an answer has not yet been officially accepted. For posterity, I figured I would try to answer it to the best of my ability in the way that the question has been phrased.

First, it is important to have a broad understanding of what a stochastic process is. It is a fairly simple concept, analogous to a random variable. However, where the value of a random variable can take on certain numbers with various probabilities, the "values" of stochastic processes manifest as certain waveforms (again, with various probabilities). As an example in the discrete world, the outcome of a coin flip could be viewed as a random variable - it can take on two values with roughly equal probability. However, if you recorded the outcome of n coin flips (where n could be any whole number, up to infinity), and were to do so many times, you could view this "set of n coin flips" as a stochastic process. Results where roughly half are heads and half tails would have relatively high probabilities, while results where almost all are heads or almost all tails would have relatively low probabilities. Obviously, there are also continuous random variables and stochastic processes can be either discrete or continuous for both axes (time/trials vs. values/outcomes of each trial).

It is also important to understand expected value. This is even simpler - it's the value that, over a long period of time/many trials, you would expect your random variable to have. It's the mean. The average. Integrate/sum over all time/trials and divide by the amount of time/number of trials.

Now that these two things are covered, the ensemble average of a stochastic process can be explained in simple language and mathematical terms. In the simplest sense, the ensemble average is analogous to expected value. That is, given a large number of trials, it is the "average" waveform that would result from a stochastic process. Note that this means that an ensemble average is a function of the same variable that the stochastic process is. Mathematically, it can be denoted as:

$$E[X(t)] = \mu_X(t) = \int_{-\infty}^\infty x*p_{X(t)}(x)dx$$

where $p_{X(t)}$ is the PDF of $X(t)$.

You also mentioned the time average for a stochastic process. This is a very different thing, which itself is actually a random variable! The reason for this is that a time average of a stochastic process is simply the average value of a single outcome of a stochastic process. Note that this means that unlike the ensemble average, the time average is not a function, but a value (a number). It can be described mathematically as:

$$\lim_{T\to\infty} \frac{1}{2T}\int_{-T}^T X(t)dt$$

where $X(t)$ is the stochastic process in question, evaluated at time $t$.

To wrap up: ensemble and time averages are properties of stochastic processes, which are like random variables but take the form of waveforms. Ensemble average is analogous to expected value or mean, in that it represents a sort of "average" for the stochastic process. It is a function of the same variable as the stochastic process, and when evaluated at a particular value denotes the average value that the waveforms will have at that same value. Time average is more like a typical average, in that it is the average value of a single outcome of a stochastic process. It is a random variable itself, as it depends upon which outcome it is being evaluated for (and the outcome itself is random).

• VerumCH, great and clear answer. Would you remember from which source(s) you consulted in writing this? Jul 14 '19 at 23:39
• @flow2k - at the time I answered this question I was in the middle of a statistics course in college (and just to happened to be recently involved with stochastic processes and related topics). A lot was from memory; the formulas and original info came primarily from the textbook we had for that class, which I don't remember the name of. Jul 14 '19 at 23:54
• Would you mind adding a short sentence or two on ergodicity? Is it true that if the underlying process is ergodic then the time averages are equal to the ensemble averages? And can you define an "expectation" value for the time average case as well? Apr 6 '20 at 14:53
• Also, I don't understand why you say that the time average is just a number. If you have $N$ members in the ensemble, then you would obtain $N$ time averages - one for each waveform - you would have a set of numbers. In the same way, the ensemble average should give you a single number for each time $t_i$ after you average over $N$ realisations, and then when you collect all times $t_i$ together you would also form a set of numbers. I don't see why there is a distinction between the ensemble average being a function and the time average being a number.. Apr 6 '20 at 20:32

for analog continous signals, we have time average.Time average is averaged quantity of a single system over a time inetrval directly related to a real experiment. or discrete signals, we have ensemble average. Ensemble average is averaged quantity of a many identical systems at a certain time.

EDIT: A clearer example of this is given on the Wikipedia page on Ergodicity.

"Each resistor has thermal noise associated with it and it depends on the temperature. Take N resistors (N should be very large) and plot the voltage across those resistors for a long period. For each resistor you will have a waveform. Calculate the average value of that waveform. This gives you the time average. You should also note that you have N waveforms as we have N resistors. These N plots are known as an ensemble. Now take a particular instant of time in all those plots and find the average value of the voltage. That gives you the ensemble average for each plot."

• Do you think that this is an answer to the question ? May 20 '16 at 6:20
• I think there is mentioned something "Could you please give me a simple definition with some examples for ensemble averaging? Compare time averaging and ensemble averaging?".No offence . Well, I tried to answer It may not be appropriate. But I think answer can be interpreted by this example. Well, Sir,If I am wrong you are most welcome to correct this answer or give something understandable to this question.
– subh
May 20 '16 at 6:52

The output of a random experiment is generaly treated as a random variable, and we know the definition of the mean (expected value) of a random variable. But in a more general setup, like for example a stochastic process (its just a name, nothing complex about it) the output of a stochastic process is a more general object rather than just a random number. The ensemble is defined as a set of all possible outcomes of a stochastic process, and ensemble average means the expected object (like expected value for random variable) of the stochastic process. Simply speaking it is just the expected value of random variable, but defined for a more general abstract setup.

• I think this is a bit misleading at least in terms of physics terminology; an ensemble is a large collection of representatives drawn from phase space according to the distribution on the phase space. But it is not the phase space itself.
– Ian
Jun 25 '15 at 16:17
• @Ian : quoting OP : "not only in this context but the exact meaning and use of ensemble averaging in statistics and mathematics." Jun 25 '15 at 16:20
• The notion of phase space is not restricted to the physical context. It originated there, but it is standard terminology elsewhere now as well.
– Ian
Jun 25 '15 at 16:21
• @Ian : Sorry, i am an engineer by training in electronics and communication systems, we study stochastic processes, and have never heard of the term 'phase space'. Jun 25 '15 at 16:23
• If you've done anything with Hamiltonian mechanics, you've interacted with the idea even if you've never heard the term. In Hamiltonian mechanics on $N$ particles in 3 dimensions, the phase space is a subset of $\mathbb{R}^{6N}$, since each particle has 3 position coordinates and 3 momentum coordinates. The situation is similar in Lagrangian mechanics.
– Ian
Jun 25 '15 at 16:24