# Combining two probability distributions

I have a variable $X$. In a measurement $A$, $X$ follows the normal distribution $N_1$ with mean $m_1$ and standard deviation $\sigma_1$. In a similar measurement $B$, $X$ follows another normal distribution $N_2$ with mean $m_2$ and standard deviation $\sigma_2$. In this case, what will be the combined or joint probability distribution of $X$? Will it be $N_1+N_2$ or $N_1 N_2$ ?

(Addition) Let's assume $A$ and $B$ are independent measurements. We can think about a situation when a measurer $A$ comes and measure the distribution of $X$ and then next a person $B$ comes and measures the distribution again. Both measurers measure independently. The question is what will be the true probability distribution of $X$ in this case? We assume that the measurement of $A$ and $B$ are equally reliable.

(Paraphrasing a comment from r.e.s.) If person C receives reports from equally-reliable observers A and B, stating their respective independent judgements about X (in the form of the stated normal distributions), then how does C combine these reports to form a fair and unbiased judgement about X?

-
I have a couple of questions: 1. what is $N_1\times N_2$? 2. what are these measurements and how do they define the joint distribution of a random variable? –  Ilya Dec 2 '11 at 21:29
Normally the joint probability distribution of two random variables is specified by a function of two variables, often a cumulative probability distribution function or a probability density function. It's not the distribution of $N_1+N_2$ or $N_1 N_2$ or the like; it's the distribution of $(N_1,N_2)$. And you haven't given enough information. For example: are these observations independent, and if not, what is the nature of the dependence? –  Michael Hardy Dec 2 '11 at 23:39
Thanks Ilya and Michael Hardy. I have clarified in the post. –  Finn Larsen Dec 3 '11 at 7:21
Have you?   –  Did Dec 3 '11 at 7:43
Somehow your two "measurers" are said to independently conclude that $X$ has two different normal distributions. What's missing is an explanation of how they do that. Is each measurer obtaining different (finite) samples having the given sample means & variances? What are the sample sizes? –  r.e.s. Dec 3 '11 at 14:03

The work by Clemen & Winkler is not for this situation. For example, if $P=(1,0,0,0)$ is a probability distribution over a $4$-element set, and $Q=(0.5, 0.3, 0.2, 0)$ is another independently obtained probability distribution over that set, then the probability distribution, $F(P,Q)$, resulting from combining information in $P$ and $Q$, should be $(1,0,0,0)$ because $P$ already has conclusive information on the set elements, that cannot be further "improved" by another observation. In other words, any $0$-value occurring in $P$ (or $Q$) must result in a $0$-value in $F(P,Q)$ at the same position (the same for any $1$-value logically follows from this). Also, the identity of $F$ should be the uniform distribution $(0.25, 0.25, 0.25, 0.25)$, as that is the most inconclusive distribution. Aggregating $P$ and $Q$ by taking their weighted arithmetic or geometric mean does not achieve this, as in most works like Clemen & Winkler. Could someone please suggest how such a function $F$ should be defined? We must assume that $P$ and $Q$ are consistent, i.e. if one of them has a $0$-value in a certain position, then the other does not have a $1$-value in the same position, because our two observations cannot contradict each other. I am not an expert in probability theory, so please pardon any inaccurate terminology usage. I would appreciate any help with finding such an $F$. Thanks!

-

The quantity you request is the joint probability distribution $P(x_A,x_B)$, that is the probability that $A$ observes $x_A$ while at the same time $B$ observes $x_B$. You've specified that observer $A$ sees a normal distribution, in other words you're saying $$\int P(x_A,x_B) dx_B = N(x_A;m_A,\sigma_A) = \frac{1}{\sigma_A\sqrt{2\pi}} e^{-\frac{(x_A-m_A)^2}{2\sigma_A^2}}$$ then you tell us that observer B sees a similar result with possibly different mean and sigma: $$\int P(x_A,x_B) dx_A = N(x_B;m_B,\sigma_B) = \frac{1}{\sigma_B\sqrt{2\pi}} e^{-\frac{(x_B-m_B)^2}{2\sigma_B^2}}$$ where I have used the subscripts $A,B$ instead of $1,2$ to indicate the respective observations.

The assumption that the observations are statistically independent means that $$P(x_A,x_B) = P(x_A)\,P(x_B),$$ which trivially satisfies the above integral relationships as $N_1,N_2$ are normalized. So to answer your initial question, the correct answer is $N_1 N_2$. One never sums probability distributions unless the observations are mutually exclusive. In other words $N_1+N_2$ could only have been the answer if when $A$ makes an observation, there is no corresponding observation from $B$. One can see there's a further problem that the sum isn't even normalized, so that could not have been the answer.

Because you specified the problem in terms of integral relationships, there are other possible answers if statistical independence is not assumed. Consider for example a multivariate normal distribution, which can satisfy the above integral relationships though it is not separable. One can further make the multivariate normal distribution elliptical, as long as the projections onto the two coordinate axes are held fixed. In other words, there could be an off-diagonal correlation matrix. One should always check the correlation matrix for real measurements, because sometimes you can have a very precise measurement in an off-diagonal direction like $x_A+x_B$ but very large errors in the orthogonal direction ($x_A-x_B$), and this would show up as mediocre errors on both axes when projected onto the coordinate axes.

-

Quoting an above comment by r.e.s.:

An example of the literature is "Aggregating Probability Distributions" by R. Clemen & R. Winkler. (This may be the same as Ch. 9 in "Advances in decision analysis: from foundations to applications" by Ward Edwards, Ralph F. Miles, Detlof Von Winterfeldt.) Note in particular the linear aggregation pool approach (p. 7).

-