Why does marginalization of a joint probability distribution use sums? I'm going through a book that talks about probability distributions. The part which is tripping me up is conceptual. It says:
"We can recover the probability distribution of any single variable from a joint
distribution by summing (discrete case) or integrating (continuous case) over all
the other variables."
Why is this? What does this mean? 
I know this is a very open ended question - but I really don't understand at a conceptual level what this means. 
To be a bit more clear:
Imagine a joint distribution Pr(x, y) of two discrete distributions Pr(x) and Pr(y) as a Hinton diagram - where Pr(x) is horizontally and Pr(y) is vertically represented. Say there are 5 values in both directions. 
To get x then according to the above statement - do I add the first column and that would give me the first value of Pr(x) and so on? If so, why? If not, what is the right way?
 A: Suppose $X$ can be either $1$, $2$, $3$ or $4$, and $Y$ can be either $1$, $2$, or $3$.  What is $\Pr(X=1)$?  It is a marginal probability. And it is
\begin{align}
\Pr(X=1) & = \Pr \Big( (X=1 \text{ and } Y=1)\text{ or }(X=1 \text{ and }  Y=2) \text{ or }(X=1 \text{ and }  Y=3) \Big) \\[10pt]
& = \Pr(X=1\ \text{and }  Y=1) + \Pr(X=1\ \text{and }  Y=2) + \Pr(X=1\ \text{and }  Y=3).
\end{align}
This is a sum of values of the joint probability distribution.
A: If $I$ is countable and rv $Y$ only takes values in $\{y_i\mid i\in I\}$ then: $$\{X=x\}=\bigcup_{i\in I}\{X=x\wedge Y=y_i\}$$
The sets $\{X=x\wedge Y=y_i\}$ are disjoint so consequently: $$P\{X=x\}=\sum_{i\in I}P\{X=x\wedge Y=y_i\}$$
If $Y$ is continuously distributed then the sum becomes an integral.
This also works if more generally $\{X=x\}$ is replaced by $\{X\in A\}$
A: First, consider the definition of the probability distribution or cumulative distribution function or simply distribution function of a one-dimensional random variable or single event as
The probability of the event $\{X\leq x\}$ is called a probability distribution of random variable $X$ and is denoted by $F_{X}(x)$ and stated as:
$$ 
F_{X}(x)=P(X\leq x)  \hspace{1cm}  for -\infty\leq x \leq \infty
$$
In other words $F_{X}(x)$ is the probability that $X$ takes any value in the range $(-\infty,x)$.
Now suppose A and B are events defined by the correspondence 
$ A =\{X|X \leq x\}$ and $ B =\{Y|Y \leq y\}$ then genetically A corresponds to the set of all points to the left of the vertical line $X=x$ and B corresponds to the set of points below horizontal line $Y=y$.If $ A\cap B$ is the intersection of the two events then it's mapping will be a common area as shown in the 
fig here 
Now consider the definition of the probability distribution or  Distribution function for the two-dimensional random variable $(X, Y)$ as 
The probability of the joint event $\{X\leq x, Y\leq y\}$ is called a Joint probability distribution of random variable $X$ and $Y$ is denoted by $ F_{X,Y}(x,y)$ and stated as:
$$
F_{X,Y}(x,y)=P(X\leq x,Y\leq y)
$$
In other words if the events defined as above i.e. if $ A =\{X|X \leq x\}$ and $ B =\{Y|Y \leq y\}$ then $ A \cap B = \{(X,Y)|X\leq x,Y\leq y\}$ and we have 
$$
F_{X,Y}(x,y)=P(X\leq x,Y\leq y)=A \cap B
$$
Now when we set $y$ to $\infty$,this is equivalent to making $B$ a certain event i.e. Y taking any from $-\infty$ to $\infty$ i.e. $ B =\{Y|Y \leq \infty\}=S$. where $S$ is sample space.So we have $A\cap B =A\cap S=A$ 
$\therefore \hspace{0.5cm}  F_{X,Y}(x,\infty)=P(X\leq x,Y\leq \infty) \\= P(A \cap S) = P(A)\\= P(X \leq x) = F_{X}(x)$
That is nothing but the Distribution Function or Probability Distribution for the single event $A$
Now consider in terms of the simple definition of the Joint Probability 
$$
P(A/B)=\frac{card(A \cap B)/card(S)}{car(B)/card(S)}=P(A \cap B)/P(B)
$$
Again with the same reason if we Marginalized or made powerless to $B$ as $B =\{Y|Y \leq \infty\}=S$ we left with $P(A)$.
