0
$\begingroup$

I noticed something peculiar when I define the binomial distribution: $$p(r,n,\theta)=\theta^r(1-\theta)^{n-r} {n\choose r}$$ we can look at this distribution as a function of $r$, the sampling distribution: $s_{n,\theta}(r)=p(r,n,\theta)$, or as the likelihood of $\theta$: $L_{r,n}(\theta)=p(r,n,\theta)$.

What I noticed is that if we set $n$ small, these functions resemble each other somewhat though they are not the same, but when we set n very large, these functions tend to exactly the same values: $$s_{n,a}(x)\approx L_{a,n}(x)$$


Then I noticed that in the case of the normal distribution $\frac 1 {\sqrt {2\pi \sigma ^2}}e^{-\frac {(x-\mu)^2} {2 \sigma ^2}}$, the sampling distribution and the likelihood of $\mu$ are exactly the same, since we can just swap $x$ and $\mu$.

Of course in the normal distribution this is obvious from the formula, though I never explicitly thought about this in terms of the sampling distribution being equal to the likelihood function.

But in the case of the binomial distribution, it is not at all obvious to me from the formula why sampling distribution = likelihood function.


More generally, is there a general principle that the likelihood function and the sampling distributions should equal each other? Is there an intuitive reason why this should be the case?

$\endgroup$
0
$\begingroup$

Because of the definition of the likelihood function, which is defined as the joint probability of your sample observations (cf. Wackerly, Mathematical Statistics With Applications).

If your observations are independant from each other, then the joint probability is the product of the probabilities (cf. Bayes https://stats.stackexchange.com/questions/21822/understanding-naive-bayes/21849#21849).

So in the end you just have a product of probability density functions.

But to answer your question, clearly the sampling distribution of a statistic function of your sample is not always equal to the likelihood function. Take for example the normal distribution : the likelihood will be : $$ L(\mu, \sigma^2 ) = f(y_1, y_2, ...,y_n |\mu, \sigma^2) = \left(\frac{1}{2\pi\sigma^2}\right)^{n/2}e^{\frac{-1}{2\sigma^2}\sum_{i=1}^{n}(y_i-\mu)^2}$$ Now compare it to the probability density of the sampling distribution of $\bar{Y},$ the sample mean of variables $ \sim N(\mu,\sigma^2) $ :$$ f(y) = \frac{1}{\frac{\sigma}{\sqrt{n}}\sqrt{2\pi}}e^{-\frac{(y-\mu)^2}{2\sigma^2}}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.