Binomial distribution/conjugate posterior

Let's consider some experiment with tossing a coin. NOTE: my question is given at the very last paragraph.

Observation $$y=0$$ or $$y=1$$ [tails (T) or heads (H)], $$p \in [0, 1]$$ (probability of heads)

We're going to look at an example where we update our belief about the probability that a coin will show HEADS, starting from before our first flip, and update our belief every time we flip a coin, and see a new observation.

Let's say you start off agnostic on probability of seeing H. The prior is $$Beta(1, 1)$$. Now you flip the coin 3 times, and observe the sequence {H, H, T}.

Starting with a prior $$Beta(1, 1)$$, you flip and observe an H. Combining your prior with the likelihood, your posterior is now $$Beta(2, 1)$$.

Now your posterior becomes your prior, $$Beta(2, 1)$$, you flip a second time, and observe an H, your posterior is now $$Beta(3, 1)$$.

Now your posterior becomes your prior, $$Beta(3, 1)$$, you flip a second time, and observe an T, your posterior is now $$Beta(3, 2)$$.

The idea here is that you have some prior belief about what the probability of getting H (heads) is, prior to ever flipping a coin. It may be that you are neutral, $$a=1$$, $$B=1$$, that is all $$p$$ equally likely (you have no idea). When you flip the coin once, this gives you some new information. Let's say you flip and get an H. Now this boosts the probability of higher values of $$p$$.

Note that as soon as you see your first Tail after the 3rd flip, the prior probability of p is now 0 at p=1 - ie there is SOME probability of seeing Tail. QUESTION: I just wonder what would happen if in the 4th picture I would add a point above $$p=1$$ OR small curve above the interval (p-epsilon,p],RESPECTIVELY. Would it be still the case that "there is SOME probability of seeing Tail."? ALSO, WHICH experiment could yield this 2 modified pictures/data??

• "add a point above $p=1$"??? – David G. Stork Dec 31 '18 at 18:50
• Yes, in the r.h.s. 4th picture not value $0$ but a ($\frac{1}{2}$-multiple) of the delta function ($\Delta$ ).The distribution will be discontinuous there. And we will modify (lessen to one half) the graph at the l.h.s. accordingly so that the inetgral of the entire curve will be 1. – user122424 Dec 31 '18 at 18:51
• Oh... do you mean instead "add a point at $p = 1$? The vertical axis doesn't mean having a point "above" a value. – David G. Stork Dec 31 '18 at 19:06
• I mean add a point/Dirac's $\Delta$ above the very right hand side place of the horizontal axe (i.e.at value $p=1$). – user122424 Dec 31 '18 at 19:18