Fun Inference with Low Number of Data Points. Apples on my large orchard can either be red or green, I have observed that two apples are red and seen no other apples, what does this tell me about that ratio of red to green apples?
Thanks
 A: The beta family of distributions is commonly used for prior
distribution of the binomial success probability $\theta$.
The first reason is that $(0,1)$ is the support for beta
distributions. The second is that the beta family has many
different shapes. The third is that beta priors are 'conjugate'
to (mathematically compatible with) binomial likelihoods, thus
making distributional computations easy.
Let Success be getting a red apple on a random draw from
your orchard and suppose your prior density is $p(\theta)$ is $Beta(3,3),$ which
means you believe the ratio may be 'somewhere near' 50:50, but
not necessarily 'very close'. In particular $P(.2 < \theta <.8) \approx 0.88.$  Also, as we begin a 90% Bayesian probability
interval for $\theta$ is $(.189, .811).$ In R:
 diff(pbeta(c(.2,.8), 3, 3))
 ## 0.88416
 qbeta(c(.05,.95), 3, 3)
 ## 0.1892554 0.8107446

Then your binomial likelihood function based on 2 successes out of 2
is $$p(x|\theta) \propto \theta^2 (1 - \theta)^{2-2} = \theta^2,$$
where the symbol $\propto$ indicates that we have omitted
the unnecessary binomial coefficient.
Then Bayes' Theorem says
$$\text{POSTERIOR}  \propto \text{PRIOR} \times \text{LIKELIHOOD}$$
or 
$$p(\theta|x) \propto p(\theta)p(x|\theta) 
\propto \theta^{3-1}(1 - \theta)^{3-1} \times \theta^2 
= \theta^{5-1}(1-\theta)^{3-1},$$
where we recognize at the right, the kernel of $Beta(5,3).$
Thus, our posterior 90% probability interval for $\theta$ is
$(.341, .871)$---slightly more optimistic about
getting red apples from your orchard than before, but then
two observed red apples don't count for a lot if your prior
is even mildly concentrated at '50:50-ish',
 qbeta(c(.05,.95), 5, 3)
 ## 0.3412614 0.8712436

Note: If we had begun with a noninformative prior $Beta(.5, .5)$
our 90% interval would have gone from prior $(.006, .994)$ to posterior$(.431, .999)$. 
 qbeta(c(.05,.95), .5, .5)
 ## 0.00615583 0.99384417
 qbeta(c(.05,.95), 2.5, .5)
 ## 0.4307415 0.9991318

When we have very little data, the prior distribution makes
a big difference in the posterior. So, indeed as suggested
by @algamest, it really does depend on your prior distribution.
However, an advantage of applied
Bayesian statistics is that yesterday's posterior can be today's
prior. So you can refine your opinion about the proportion of red apples in your
orchard as you continue to (randomly) collect and munch apples over time.
