Questions tagged [statistics]

Mathematical statistics is the study of statistics from a mathematical standpoint, using probability theory and other branches of mathematics such as linear algebra and analysis.

332
votes
4answers
165k views

What is the intuitive relationship between SVD and PCA?

Singular value decomposition (SVD) and principal component analysis (PCA) are two eigenvalue methods used to reduce a high-dimensional dataset into fewer dimensions while retaining important ...
150
votes
7answers
108k views

How to intuitively understand eigenvalue and eigenvector?

I'm learning multivariate analysis and I have learnt linear algebra for two semester when I was a freshman. Eigenvalue and eigenvector is easy to calculate and the concept is not difficult to ...
109
votes
6answers
28k views

How often does it happen that the oldest person alive dies?

Today, we are brought the sad news that Europe's oldest woman died. A little over a week ago the oldest person in the U.S. unfortunately died. Yesterday, the Netherlands' oldest man died peacefully. ...
101
votes
9answers
28k views

If I flip a coin 1000 times in a row and it lands on heads all 1000 times, what is the probability that it's an unfair coin?

Consider a two-sided coin. If I flip it $1000$ times and it lands heads up for each flip, what is the probability that the coin is unfair, and how do we quantify that if it is unfair? Furthermore, ...
91
votes
6answers
58k views

derivative of cost function for Logistic Regression

I am going over the lectures on Machine Learning at Coursera. I am struggling with the following. How can the partial derivative of $$J(\theta)=-\frac{1}{m}\sum_{i=1}^{m}y^{i}\log(h_\theta(x^{i}))+...
68
votes
9answers
7k views

What's so special about standard deviation?

Equivalently, about variance? I realize it measures the spread of a distribution, but many other metrics could do the same (e.g., the average absolute deviation). What is its deeper significance? ...
65
votes
4answers
111k views

What is the difference and relationship between the binomial and Bernoulli distributions?

How should I understand the difference or relationship between binomial and Bernoulli distribution?
61
votes
4answers
100k views

Variance of sample variance?

What is the variance of the sample variance? In other words I am looking for $\mathrm{Var}(S^2)$. I have started by expanding out $\mathrm{Var}(S^2)$ into $E(S^4) - [E(S^2)]^2$ I know that $[E(S^2)]^...
59
votes
3answers
22k views

Intuitive explanation of a definition of the Fisher information

I'm studying statistics. When I read the textbook about Fisher Information, I couldn't understand why the Fisher Information is defined like this: $$I(\theta)=E_\theta\left[-\frac{\partial^2 }{\...
58
votes
17answers
13k views

Is the Law of Large Numbers empirically proven?

Does this reflect the real world and what is the empirical evidence behind this? Layman here so please avoid abstract math in your response. The Law of Large Numbers states that the average of the ...
55
votes
4answers
34k views

Intuition behind using complementary CDF to compute expectation for nonnegative random variables

I've read the proof for why $\int_0^\infty P(X >x)dx=E[X]$ for nonnegative random variables (located here) and understand its mechanics, but I'm having trouble understanding the intuition behind ...
55
votes
11answers
23k views

Why do we use a Least Squares fit?

I've been wondering for a while now if there's any deep mathematical or statistical significance to finding the line that minimizes the square of the errors between the line and the data points. If ...
54
votes
6answers
24k views

incremental computation of standard deviation

How can I compute the standard deviation in an incremental way (using the new value and the last computed mean and/or std deviation) ? for the non incremental way, I just do something like: ...
50
votes
4answers
29k views

how does expectation maximization work?

I'm reading a tutorial on expectation maximization which gives an example of a coin flipping experiment (the description is at http://www.nature.com/nbt/journal/v26/n8/full/nbt1406.html?pagewanted=all)...
46
votes
4answers
30k views

How was the normal distribution derived?

Abraham de Moivre, when he came up with this formula, had to assure that the points of inflection were exactly one standard deviation away from the center, and so that it was bell-shaped, as well as ...
46
votes
3answers
18k views

Is there a mathematical reason why chocolate chip cookies have 37% (1/e) chocolate in them?

Someone once briefly explained to me why it is that chocolate chip cookies have 37% chocolate in them. To the best of my memory it has to do with the way trying to place dots in a circle in a random ...
45
votes
1answer
119k views

Sample Standard Deviation vs. Population Standard Deviation

I have an HP 50g graphing calculator and I am using it to calculate the standard deviation of some data. In the statistics calculation there is a type which can have two values: Sample Population I ...
44
votes
2answers
17k views

Why is the error function defined as it is?

$\newcommand{\erf}{\operatorname{erf}}$ This may be a very naïve question, but here goes. The error function $\erf$ is defined by $$\erf(x) = \frac{2}{\sqrt{\pi}} \int_0^x e^{-t^2}dt.$$ Of ...
43
votes
10answers
9k views

Real life usage of Benford's Law

I recently discovered Benford's Law. I find it very fascinating. I'm wondering what are some of the real life uses of Benford's law. Specific examples would be great.
41
votes
3answers
58k views

density of sum of two uniform random variables $[0,1]$

I am trying to understand an example from my textbook. Let's say $Z = X + Y$, where $X$ and $Y$ are uniform random variables with range $[0,1]$. Then the PDF is $$f(z) = \begin{cases} z & \text{...
40
votes
3answers
6k views

Why does this not seem to be random?

I was running a procedure to be like one of those games were people try to guess a number between 1 and 100 where there are 100 people guessing.I then averaged how many different guesses there are. <...
38
votes
4answers
835 views

How does a disease spread through a triangular network?

Consider a population of nodes arranged in a triangular configuration as shown in the figure below, where each level $k$ has $k$ nodes. Each node, except the ones in the last level, is a parent node ...
37
votes
4answers
84k views

Why the sum of residuals equals 0 when we do a sample regression by OLS?

That's my question, I have looking round online and people post a formula by they don't explain the formula. Could anyone please give me a hand with that ? cheers
34
votes
10answers
4k views

Is it possible to have 2 different but equal size real number sets that have the same mean and standard deviation?

By inspection I notice that Shifting does not change the standard deviation but change mean. {1,3,4} has the same standard deviation as ...
34
votes
2answers
25k views

Why we consider log likelihood instead of Likelihood in Gaussian Distribution

I am reading Gaussian Distribution from a machine learning book. It states that - We shall determine values for the unknown parameters $\mu$ and $\sigma^2$ in the Gaussian by maximizing the ...
34
votes
3answers
1k views

Guessing the length of a playlist on “shuffle random?”

The other night I was hanging out with some friends and someone put on a playlist on shuffle random, where the songs are drawn uniformly at random from a fixed playlist. The person who put the ...
33
votes
7answers
21k views

Intuitive Explanation of Bessel's Correction

When calculating a sample variance a factor of (N-1) appears instead of N (see http://en.wikipedia.org/wiki/Sample_variance#Population_variance_and_sample_variance ). Does anybody have an intuitive ...
33
votes
5answers
25k views

What does it mean to integrate with respect to the distribution function?

If $f(x)$ is a density function and $F(x)$ is a distribution function of a random variable $X$ then I understand that the expectation of x is often written as: $$E(X) = \int x f(x) dx$$ where the ...
32
votes
2answers
767 views

Statistics Primer for the Unwary Mathematician

I have a new position in a biology department (after being housed in a maths department) working on cognitive and population modeling. People in my lab are asking for help with applying statistical ...
31
votes
7answers
10k views

Why does Benford's Law (or Zipf's Law) hold?

Both Benford's Law (if you take a list of values, the distribution of the most significant digit is rougly proportional to the logarithm of the digit) and Zipf's Law (given a corpus of natural ...
31
votes
2answers
2k views

What is the difference between statistical mean and calculus mean?

For example in statistics we learn that mean = E(x) of a function which is defined as $$\mu = \int_a^b xf(x) \,dx$$ however in calculus we learn that $$\mu = \frac {1}{b-a}\int_a^b f(x) \,dx $$ ...
29
votes
2answers
137k views

Difference between Poisson and Binomial distributions. [closed]

If both the Poisson and Binomial distribution are discrete, then why do we need two different distributions?
29
votes
6answers
5k views

Motivation behind standard deviation?

Let's take the numbers 0-10. Their mean is 5, and the individual deviations from 5 are -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5 And so the average (magnitude of) ...
28
votes
2answers
95k views

maximum estimator method more known as MLE of a uniform distribution [closed]

Let $ X_1, ... X_n $ a sample of independent random variables with uniform distribution $(0,$$ \theta $$ ) $ Find a $ $$ \widehat\theta $$ $ estimator for theta using the maximun estimator ...
28
votes
6answers
2k views

Does exceptionalism persist as sample size gets large?

Which of the following is more surprising? In a group of 100 people, the tallest person is one inch taller than the second tallest person. In a group of one billion people, the tallest person is one ...
28
votes
2answers
546 views

How long does it take a person with this “cheating” data-gathering strategy to achieve a desired result?

I have a perfectly fair coin, and my goal is to prove that it is unfair with a confidence level of 95%. In order to accomplish this, I will cheat. Whenever I fail to have enough evidence, I will ...
27
votes
9answers
14k views

If a lottery has 300 tickets, shouldn't I win every 300 times I play

Suppose I play a lottery that has 300 tickets. I can only buy one ticket per draw. Statistically speaking, shouldn't I win once every 300 draws? Is it more complicated than this? Edit This question ...
27
votes
11answers
35k views

Two dice thrown, one comes up 6

If my friend throws two dice, and covers them up, but I see that one of them was a 6, what's the probability that they were both 6s given this knowledge? I'm under the impression that the answer is 2/...
27
votes
5answers
24k views

Why does zero correlation not imply independence?

Although independence implies zero correlation, zero correlation does not necessarily imply independence. While I understand the concept, I can't imagine a real world situation with zero correlation ...
27
votes
6answers
30k views

What is the use of moments in statistics?

Can anyone give me a simple explanation (i.e. without too many equations) of what is the use of moments in statistics? Why do we need moments? What can we learn from them?
26
votes
2answers
4k views

Beta function derivation

How do I derive the Beta function using the definition of the beta function as the normalizing constant of the Beta distribution and only common sense random experiments? I'm pretty sure this is ...
26
votes
4answers
34k views

Is positive the same as non-negative?

I would assume the answer to my question is yes, but I want to make sure because my book uses both terminologies. Please also indicate where zero falls into the mix. UPDATE: Here is an excerpt from ...
26
votes
5answers
12k views

Why get the sum of squares instead of the sum of absolute values?

I'm self-studying machine learning and getting into the basics of linear regression models. From what I understand so far, a good regression model minimizes the sum of the squared differences between ...
25
votes
12answers
7k views

Statistics: Why doesn't the probability of an accurate medical test equal the probability of you having disease?

Suppose there is a test for Disease A that is correct 90% of the time. You had this test done, and it came out positive. I understand that the chance that this test is right is 90%, but I thought this ...
25
votes
5answers
42k views

What is degree of freedom in statistics?

In statistics, degree of freedom is widely used in regression analysis, ANOVA and so on. But, what is degree of freedom ? Wikipedia said that The number of degrees of freedom is the number ...
25
votes
1answer
11k views

sum of squares of dependent gaussian random variables

Ok, so the Chi-Squared distribution with n degrees of freedom is the sum of the squares of n independent Gaussian random variables. The trouble is, my Gaussian random variables are not independent. ...
23
votes
2answers
19k views

Proof of upper-tail inequality for standard normal distribution

$X \sim \mathcal{N}(0,1)$, then to show that for $x > 0$, $$ \mathbb{P}(X>x) \leq \frac{\exp(-x^2/2)}{x \sqrt{2 \pi}} \>. $$
23
votes
7answers
42k views

How many times to roll a die before getting two consecutive sixes? [closed]

Basically, on average, how many times do you have to roll a fair six-sided die before getting two consecutive sixes?
23
votes
5answers
38k views

Is there a simple test for uniform distributions?

I have a function that (more or less) is supposed to select a small number $m$ of random numbers from the range $[1,n]$ (for some $n \gg m$) and I need to test that it work. Is there an easy to ...
23
votes
5answers
37k views

Showing that Y has a uniform distribution if Y=F(X) where F is the cdf of continuous X

Let $X$ be a random variable with a continuous and strictly increasing c.d.f. function $F$ (so that the quantile function $F^{−1}$ is well-defined). Define a new random variable $Y$ by $Y = F(X)$. Show ...