Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [probability-theory]

Use this tag only if your question is about the modern theoretical footing for probability, for example probability spaces, random variables, law of large numbers, and central limit theorems. Use [tag:probability] instead for specific problems and explicit computations. Use [tag:probability-distributions] for specific distribution functions, and consider [tag:stochastic-processes] when appropriate.

249
votes
8answers
17k views

Intuition for the definition of the Gamma function?

In these notes by Terence Tao is a proof of Stirling's formula. I really like most of it, but at a crucial step he uses the integral identity $$n! = \int_{0}^{\infty} t^n e^{-t} dt$$ coming from ...
106
votes
11answers
163k views

What is the best book to learn probability?

Question is quite straight... I'm not very good in this subject but need to understand at a good level.
90
votes
5answers
12k views

Intuition behind Conditional Expectation

I'm struggling with the concept of conditional expectation. First of all, if you have a link to any explanation that goes beyond showing that it is a generalization of elementary intuitive concepts, ...
87
votes
5answers
133k views

Is the product of two Gaussian random variables also a Gaussian?

Say I have $X \sim \mathcal N(a, b)$ and $Y\sim \mathcal N(c, d)$. Is $XY$ also normally distributed? Is the answer any different if we know that $X$ and $Y$ are independent?
81
votes
8answers
31k views

Lebesgue integral basics

I'm having trouble finding a good explanation of the Lebesgue integral. As per the definition, it is the expectation of a random variable. Then how does it model the area under the curve? Let's take ...
70
votes
4answers
98k views

Probability density function vs. probability mass function

I've a confession to make. I've been using pdf's and pmf's without actually knowing what they are. The idea that I've been having so long is that density = area under the curve but if I look at it ...
65
votes
10answers
14k views

The Monty Hall problem

I was watching the movie 21 yesterday, and in the first 15 minutes or so the main character is in a classroom, being asked a "trick" question (in the sense that the teacher believes that he'll get the ...
64
votes
6answers
17k views

Chance of meeting in a bar

Two people have to spend exactly 15 consecutive minutes in a bar on a given day, between 12:00 and 13:00. Assuming uniform arrival times, what is the probability they will meet? I am mainly ...
64
votes
14answers
21k views

Reference book on measure theory

I post this question with some personal specifications. I hope it does not overlap with old posted questions. Recently I strongly feel that I have to review the knowledge of measure theory for the ...
57
votes
3answers
22k views

Intuitive explanation of a definition of the Fisher information

I'm studying statistics. When I read the textbook about Fisher Information, I couldn't understand why the Fisher Information is defined like this: $$I(\theta)=E_\theta\left[-\frac{\partial^2 }{\...
54
votes
9answers
24k views

Proving that 1- and 2-d simple symmetric random walks return to the origin with probability 1

How does one prove that a simple (steps of length $1$ in directions parallel to the axes) symmetric (each possible direction is equally likely) random walk in $1$ or $2$ dimensions returns to the ...
52
votes
5answers
1k views

Time to reach a final state in a random dynamical system (answer known, proof unknown)

Consider a dynamical system with state space $2^n$ represented as a sequence of $n$ black or white characters, such as $BWBB\ldots WB$. At every step, we choose a random pair $(i,j)$ with $i<j$ ...
50
votes
2answers
10k views

Cardinality of Borel sigma algebra

It seems it's well known that if a sigma algebra is generated by countably many sets, then the cardinality of it is either finite or $c$ (the cardinality of continuum). But it seems hard to prove it, ...
47
votes
7answers
38k views

Zero probability and impossibility

I read a comment under this question: There are plenty of events that can occur that have zero probability. This reminds me that I have seen similar saying before elsewhere, and have never been ...
46
votes
4answers
5k views

Why is this coin-flipping probability problem unsolved?

You play a game flipping a fair coin. You may stop after any trial, at which point you are paid in dollars the percentage of heads flipped. So if on the first trial you flip a head, you should stop ...
44
votes
7answers
106k views

Poisson Distribution of sum of two random independent variables $X$, $Y$

$X \sim \mathcal{P}( \lambda) $ and $Y \sim \mathcal{P}( \mu)$ meaning that $X$ and $Y$ are Poisson distributions. What is the probability distribution law of $X + Y$. I know it is $X+Y \sim \mathcal{...
42
votes
3answers
8k views

What is the importance of the infinitesimal generator of Brownian motion?

I have read that the infinitesimal generator of Brownian motion is $\frac{1}{2}\small\triangle$. Unfortunately, I have no background in semigroup theory, and the expositions of semigroup theory I have ...
39
votes
4answers
2k views

Rain droplets falling on a table

Suppose you have a circular table of radius $R$. This table has been left outside, and it begins to rain at a constant rate of one droplet per second. The drops, which can be considered points as they ...
39
votes
1answer
46k views

Conditional and joint probability manipulations when there are 3 variables

I'm having trouble verifying why the following is correct. $$p(x, y \mid z)= p(x \mid y, z) p(y \mid z)$$ I tried grouping the $(x, y)$ together and split by the conditional, which gives me $$p(x, ...
37
votes
4answers
20k views

Are functions of independent variables also independent?

It's a really simple question. However I didn't see it in books and I tried to find the answer on the web but failed. If I have two independent random variables, $X_1$ and $X_2$, then I define two ...
36
votes
3answers
11k views

Choose a random number between $0$ and $1$ and record its value. Keep doing it until the sum of the numbers exceeds $1$. How many tries do we need?

Choose a random number between $0$ and $1$ and record its value. Do this again and add the second number to the first number. Keep doing this until the sum of the numbers exceeds $1$. What's the ...
35
votes
1answer
7k views

Formal definition of conditional probability

It would be extremely helpful if anyone gives me the formal definition of conditional probability and expectation in the following setting, given probability space $ (\Omega, \mathscr{A}, \mu ) $ ...
35
votes
2answers
6k views

What is meant by a continuous-time white noise process?

What is meant by a continuous-time white noise process? In a discussion following a question a few months ago, I stated that as an engineer, I am used to thinking of a continuous-time wide-sense-...
32
votes
11answers
16k views

Good books on “advanced” probabilities

what are some good books on probabilities and measure theory? I already know basic probabalities, but I'm interested in sigma-algrebas, filtrations, stopping times etc, with possibly examples of "...
32
votes
3answers
111k views

Determining variance from sum of two random correlated variables

I understand that the variance of the sum of two independent normally distributed random variables is the sum of the variances, but how does this change when the two random variables are correlated?
32
votes
4answers
17k views

Why does the median minimize $E(|X-c|)$?

Suppose $X$ is a real-valued random variable and let $P_X$ denote the distribution of $X$. Then $$ E(|X-c|) = \int_\mathbb{R} |x-c| dP_X(x). $$ The medians of $X$ are defined as any number $m \in \...
30
votes
2answers
39k views

Explain why $E(X) = \int_0^\infty (1-F_X (t)) \, dt$ for every nonnegative random variable $X$

Let $X$ be a non-negative random variable and $F_{X}$ the corresponding CDF. Show, $$E(X) = \int_0^\infty (1-F_X (t)) \, dt$$ when $X$ has : a) a discrete distribution, b) a continuous ...
30
votes
3answers
3k views

Random Variable Inequality

Doing a little reading over the break (The Probabilistic Method by Alon and Spencer); can't come up with the solution for this seemingly simple (and perhaps even a little surprising?) result: (A-S 1....
29
votes
3answers
1k views

Expected length of the shortest polygonal path connecting random points

$N$ points are selected in a uniformly distributed random way in a disk of a unit radius. Let $L(N)$ denote the expected length of the shortest polygonal path that visits each of the points at least ...
28
votes
3answers
19k views

Example where union of increasing sigma algebras is not a sigma algebra

If $\mathcal{F}_1 \subset \mathcal{F}_2 \subset \dotsb$ are sigma algebras, what is wrong with claiming that $\cup_i\mathcal{F}_i$ is a sigma algebra? It seems closed under complement since for all $...
28
votes
2answers
36k views

Expectation of Minimum of $n$ i.i.d. uniform random variables.

$X_1, X_2, \ldots, X_n$ are $n$ i.i.d. uniform random variables. Let $Y = \min(X_1, X_2,\ldots, X_n)$. Then, what's the expectation of $Y$(i.e., $E(Y)$)? I have conducted some simulations by Matlab, ...
28
votes
5answers
6k views

What is a Markov Chain?

What is a intuitive explanation of a Markov Chain, and how they work? Please provide at least one practical example.
28
votes
6answers
19k views

Intuitive explanation of the tower property of conditional expectation

I understand how to define conditional expectation and how to prove that it exists. Further, I think I understand what conditional expectation means intuitively. I can also prove the tower property, ...
28
votes
4answers
9k views

Weak Law of Large Numbers for Dependent Random Variables with Bounded Covariance

I'm currently stuck on the following problem which involves proving the weak law of large numbers for a sequence of dependent but identically distributed random variables. Here's the full statement: ...
28
votes
1answer
8k views

Distinguishing probability measure, function and distribution

I have a bit trouble distinguishing the following concepts: probability measure probability function (with special cases probability mass function and probability density function) probability ...
28
votes
1answer
1k views

Uniqueness of Brownian motion

May be it is a dumb question, but it vexed me a little bit. I understand the construction of the Brownian motion (first use Kolmogorov extension theorem to construct value at dyadic times and then use ...
27
votes
9answers
4k views

Is there a *simple* example showing that uncorrelated random variables need not be independent?

Is there a simple example showing that given $X,Y$ uncorrelated (covariance is zero), $X,Y$ are not independent? I have looked up two references, however, I am dissatisfied with both. In Reference ...
27
votes
4answers
3k views

Shannon entropy of a fair dice

The formula for Shannon entropy is as follows, $$\text{Entropy}(S) = - \sum_i p_i \log_2 p_i $$ Thus, a fair six sided dice should have the entropy, $$- \sum_{i=1}^6 \dfrac{1}{6} \log_2 \dfrac{1}{6}...
27
votes
4answers
14k views

Precise definition of the support of a random variable

$\newcommand{\F}{\mathcal{F}} \newcommand{\powset}[1]{\mathcal{P}(#1)}$ I am reading lecture notes which contradict my understanding of random variables. Suppose we have a probability space $(\Omega, \...
27
votes
5answers
3k views

Card doubling paradox

Suppose there are two face down cards each with a positive real number and with one twice the other. Each card has value equal to its number. You are given one of the cards (with value $x$) and after ...
26
votes
2answers
4k views

Beta function derivation

How do I derive the Beta function using the definition of the beta function as the normalizing constant of the Beta distribution and only common sense random experiments? I'm pretty sure this is ...
26
votes
3answers
19k views

Integral of Brownian motion is Gaussian?

Let $(W_t)$ be a standard Brownian motion, so that $W_t \sim N(0,t)$. I'm trying to show that the random variable defined by $Z_t = \int_0^t W_s \ ds$ is a Gaussian random variable, but have not ...
26
votes
2answers
1k views

A simple way to obtain $\prod_{p\in\mathbb{P}}\frac{1}{1-p^{-s}}=\sum_{n=1}^{\infty}\frac{1}{n^s}$.

Let $ p_1<p_2 <\cdots <p_k < \cdots $ the increasing list in set $\mathbb{P}$ of all prime numbers . By sum of infinite geometric series we have $\sum_{k=0}^\infty r^k = \frac{1}{1-r}$, ...
26
votes
3answers
5k views

Random walk on $n$-cycle

For a graph $G$, let $W$ be the (random) vertex occupied at the first time the random walk has visited every vertex. That is, $W$ is the last new vertex to be visited by the random walk. Prove the ...
26
votes
1answer
1k views

How far can probability intransitivity be stretched?

Once upon a time I read about nontransitive dice - sets of dice where "is more likely to roll a higher number than" is not a transitive relation. After the surprise wore off, I wondered - just how far ...
25
votes
3answers
6k views

Probability and measure theory

I'd like to have a correct general understanding of the importance of measure theory in probability theory. For now, it seems like mathematicians work with the notion of probability measure and prove ...
25
votes
1answer
7k views

Are vague convergence and weak convergence of measures both weak* convergence?

For quite a long time, I have been confused about the definitions of weak convergence and vague convergence of measures among other modes of convergence that root from functional analysis, mainly due ...
24
votes
2answers
19k views

Independence and conditional independence between random variables

For a family of random variables, I was wondering if independence and conditional independence under any condition among them imply each other? If not, can these two concepts imply one another under ...
24
votes
2answers
12k views

Does convergence in distribution implies convergence of expectation?

If we have a sequence of random variables $X_1,X_2,\ldots,X_n$ converges in distribution to $X$, i.e. $X_n \rightarrow_d X$, then is $$ \lim_{n \to \infty} E(X_n) = E(X) $$ correct? I know that ...
24
votes
2answers
29k views

probability density of the maximum of samples from a uniform distribution

Suppose $$X_1, X_2, \dots, X_n\sim Unif(0, \theta), iid$$ and suppose $$\hat\theta = \max\{X_1, X_2, \dots, X_n\}$$ How would I find the probability density of $\hat\theta$? Thank you!